Posts

Showing posts with the label pandas

Plotting a PANDAS DF to a 3D line-graph with changing width

Image
Plotting a PANDAS DF to a 3D line-graph with changing width I have a PANDAS DataFrame with the following data: DF0 = pd.DataFrame(np.random.uniform(0,100,(4,2)), columns=['x', 'y']) pupil_rads = pd.Series(np.random.randint(1,10,(4))) DF0["pupil_radius"] = pupil_rads DF0 [out:] x y pupil_radius 0 20.516882 15.098594 8 1 92.111798 97.200075 2 2 98.648040 94.133676 3 3 8.524813 88.978467 7 I want to create a 3D graph, showing where the gaze was pointed at (x/y coordinates) in every measurement (index of the DF). Also, I'm trying to make it a line-graph so that the radius of the line would correspond with the pupil-radius. So far what I've come up with is the following: gph = plt.figure(figsize=(15,8)).gca(projection='3d') gph.scatter(DF0.index, DF0['x'], DF0['y']) gph.set_xlabel('Time Stamp') gph.set_ylabel('X_Gaze') gph.set_zlabel('Y_Gaze') This creates...

groupby and filter pandas

groupby and filter pandas df: make country other_columns sale honda tokyo data 1 honda hirosima data 0 toyota tokyo data 1 toyota hirosima data 0 suzuki tokyo data 0 suzuki hirosima data 0 ferrari tokyo data 1 ferrari hirosima data 0 nissan tokyo data 1 nissan hirosima data 0 Condition: keep pairs that have 1,0 desired output make country other_columns sale honda tokyo data 1 honda hirosima data 0 toyota tokyo data 1 toyota hirosima data 0 ferrari tokyo data 1 ferrari hirosima data 0 nissan tokyo data 1 nissan hirosima data 0 What i tried (which is obviously not working ) outdf = df.groupby('sale').filter(lambda x: x > 0) should i be grouping both the columns ('sale' and 'make')? a pair is make + 2 state? – Anton vBR Jun 29 at 20:55 ...

Convert pandas dataframe column of UTC time string to floats

Convert pandas dataframe column of UTC time string to floats I have a pandas dataframe with a column of strings, with datetimes in UTC format, but need to convert them to floats. I'm having trouble doing this. Here is a view of my column: df['time'][0:3] 0 2018-04-18T19:00:00.000000000Z 1 2018-04-18T19:15:00.000000000Z 2 2018-04-18T19:30:00.000000000Z Name: time, dtype: object I've been trying this, but isn't working for me: import datetime for i in range(1,len(df)): df['time'][i] = datetime.datetime.strptime(df['time'][i], '%Y-%m-%dT%H:%M:%S.%f000Z') Here is the error I'm trying to fix: execfile(filename, namespace) exec(compile(f.read(), filename, 'exec'), namespace) unsup.fit(np.reshape(df,(-1,df.shape[1]))) X = _check_X(X, self.n_components) X = check_array(X, dtype=[np.float64, np.float32]) array = np.array(array, dtype=dtype, order=order, copy=copy) ValueError: could not convert string to float: '2018...

Delete column after conditional formatting (formula) using xlsxwriter

Delete column after conditional formatting (formula) using xlsxwriter I have a column (column V) that I used to conditionally format another column (column U) using engine xlsxwriter. So I have this: # Light yellow fill with dark yellow text. format1 = workbook.add_format({'bg_color': '#FFEB9C'}) # Light red fill with dark red text. format2 = workbook.add_format({'bg_color': '#FFC7CE', 'font_color': '#9C0006'}) worksheet.conditional_format('U2:U1000', {'type': 'formula', 'criteria': '=V2>25', 'format': format1}) worksheet.conditional_format('U2:U1000', {'type': 'formula', 'criteria': '=V2<-20', 'format': format2}) So now after highlighting column U with conditional formatting, ...

filter out the oldest date for a specific value in a CSV file

filter out the oldest date for a specific value in a CSV file <html> <head> <style> table { font-family: arial, sans-serif; border-collapse: collapse; width: 100%; } td, th { border: 1px solid #dddddd; text-align: left; padding: 8px; } tr:nth-child(even) { background-color: #dddddd; } </style> </head> <body> <table> <tr> <th>Item Name</th> <th>Created Date</th> <th>Other column(s)</th> </tr> <tr> <td>Item A</td> <td>6/28/2018 20:08</td> <td>not related stuff</td> </tr> <tr> <td>Item A</td> <td style="background-color:lightblue">6/28/2018 12:08</td> <td>not related stuff</td> </tr> <tr> <td>Item A</td> <td>6/28/2018 21:08</td> <td>not related stuff</td> </tr...

Pandas Convert float column containing nan values to int for merge operation

Pandas Convert float column containing nan values to int for merge operation Attempt #1 s["order_id"].apply(lambda x: int(x) if pd.notnull(x) else np.nan) Attempt #2 def to_int(x): if(pd.notnull(x)): return int(x) Attempt #3 s["order_id"] = s.loc[pd.notnull(s["order_id"]),"order_id].astype(int) All of these return a series where the values are still formatted as floats. I'm wondering if I could use the update function or take advantage of reindexing. Leveraging Indexing solution attempt: null = np.nan data = {"time":{"0":1528971021539,"1":1529289904697,"2":1529572773525,"3":1529892602301,"4":1530082881098,"5":1530069453264,"6":1528985491630,"7":1529236762719,"8":1529475504491,"9":1529814085541,"10":1529906568681,"11":1530160346468,"12":1529833559160,"13":1530051985183,"14":153024...