pandas is a powerful library in python, so I won't rehash it here, but I'll briefly introduce pandas data filtering with date ranges.
date conversion
The columns used for filtering are of type date, so here you need to convert the date range to be filtered from string to date
For example, my data contains the column name trade_date, from 20050101 - 20190926, I want to filter out the data from 20050606 - 20071016, then, first of all, as follows to convert the data type:
s_date = ('20050606', '%Y%m%d').date() e_date = ('20071016', '%Y%m%d').date()
Data filtering
Very simple, one line of code:
df = df[(df['tra_date'] >= s_date) & (df['tra_date'] <= e_date)]
caveat
- When multiple filters co-exist, they cannot be joined by and, but by a single & symbol.
- s_date <= df['trade_date'] <= e_date Equivalent to and
Five ways pandas extracts data for a certain time range
import pandas as pd #Read the file df = pd.read_csv('./') # A few ways to get September data #Method 1 Use row index slicing, ['2019/9/1':'2019/9/30'], with the disadvantage of requiring the dates to be consecutive. For ease of viewing take the first 5, all other methods below take the first 5, the order will vary as they are not sorted df.set_index('Date',inplace=True) print(df['2019/9/1':'2019/9/30'].head()) #orprint(['2019/9/1':'2019/9/30',:]) ''' Print: Maximum Temperature Minimum Temperature Weather Wind Direction Wind Level Air Quality Date 2019/9/1 33℃ 19℃ Cloudy~Sunny Southwest Wind Level 2 Good 2019/9/2 34℃ 20℃ Sunny Southeasterly wind Grade 2 Good 2019/9/3 33℃ 20℃ Sunny Southeast wind Grade 2 Good 2019/9/7 34℃ 21℃ Sunny Southwest wind 2 step (of stairs) very much 2019/9/8 35℃ 22℃ clear~cloudy Northeast wind 2 step (of stairs) very much ''' #Method 2 Generate bool list using list generator style and startwith('2019/9'),Disadvantage, more cumbersome. print([[True if ('2019/9') else False for i in ()],:].head()) ''' Print: Maximum Temperature Minimum Temperature Weather Wind Direction Wind Level Air Quality Date 2019/9/4 32℃ 19℃ Sunny Southeast wind Grade 2 Good 2019/9/5 33℃ 20℃ Sunny Southeast wind Grade 2 Good 2019/9/6 33℃ 20℃ Sunny Southeast wind Grade 1 Good 2019/9/1 33℃ 19℃ Cloudy~sunny Southwest wind 2 step (of stairs) very much 2019/9/2 34℃ 20℃ Sunny Southeasterly wind 2 step (of stairs) very much ''' #Method 3 Utilize str and startswith('2019/9')|contains('2019/9') in pandas. df1 = pd.read_csv('./') print(df1[df1['Date'].('2019/9')].head()) ''' Print: Date Maximum Temperature Minimum Temperature Weather Wind Direction Wind Level Air Quality 243 2019/9/4 32℃ 19℃ Sunny Southeast Wind Level 2 Good 244 2019/9/5 33℃ 20℃ Sunny Southeast wind Grade 2 Good 245 2019/9/6 33℃ 20℃ Sunny Southeast wind Grade 1 Good 246 2019/9/1 33℃ 19℃ Cloudy~sunny Southwest wind 2 step (of stairs) very much 247 2019/9/2 34℃ 20℃ Sunny Southeasterly wind 2 step (of stairs) very much ''' #Method 4: Convert a date to datetime. df1['Date'] = pd.to_datetime(df1['Date']) df1.set_index('Date',inplace=True,drop=True) #print(df1['2019']) # take 2019 data, or ['2019'] print(df1['2019/09'].head()) ''' Fetch the data of month 201909, other morphing writing df['2019-9'] df['2019-09'] df['2019/9'] ['2019-9',:] ['2019-09',:] ['2019/09',:] ['2019/9',:] Print: Maximum Temperature Minimum Temperature Weather Wind Direction Wind Level Air Quality Date 2019-09-04 32℃ 19℃ Sunny Southeast wind Grade 2 Good 2019-09-05 33℃ 20℃ Sunny Southeast wind Grade 2 Good 2019-09-06 33℃ 20℃ Sunny Southeast Wind Grade 1 Good 2019-09-01 33℃ 19℃ Cloudy~Sunny Southwest wind 2 step (of stairs) very much 2019-09-02 34℃ 20℃ Sunny Southeasterly wind 2 step (of stairs) very much ''' #Note that if you want to get data for a particular day, you must use slicing, e.g. df['2019/9/1':'2019/9/1'] ''' Getting some time (after = '2019-9-01') # Returns after previous data (before = '2019-9-01') # Returns data from before df1['20190901':'2019/9/10'] ''' #Method 5 # When reading a file, convert the date to datetime by parse_dates=['date'], which is equivalent to pd.to_datetime, and use index_col to index that column as a row, which is equivalent to set_index. df2 = pd.read_csv('./',parse_dates=['Date']) df2['Year'] = df2['Date']. df2['Moon'] = df2['Date']. qstr = "surname Nian=='2019' and moon=='9'" print((qstr).head()) ''' Print: Date Maximum Temperature Minimum Temperature Weather Wind Direction Wind Level Air Quality Year Month 243 2019-09-04 32℃ 19℃ Clear Southeast Wind Level 2 Good 2019 9 244 2019-09-05 33℃ 20℃ Clear Southeast Wind Level 2 Good 2019 9 245 2019-09-06 33℃ 20℃ Sunny Southeast Wind Grade 1 Good 2019 9 246 2019-09-01 33℃ 19℃ Cloudy~sunny Southwest wind 2 step (of stairs) very much 2019 9 247 2019-09-02 34℃ 20℃ Sunny Southeasterly wind Grade 2 Good 2019 9 ''' ''' Other common properties and methods of dt are listed below: df['Date']. # Date of withdrawal df['Date']. # Year of extraction df['Date']. # of hours of extraction df['Date']. # of minutes of extraction df['Date']. # Extracted in seconds df['Date']. # Week of the year df['Date']. # Returns the day of the week, with 0 being Monday and 6 being Sunday. df['Date']. # Return to the day of the year df['Date']. # Get the quarter for each date. df['Date'].dt.is_month_start # Determine if the date is the first day of the month df['Date'].dt.is_month_end # Determine if the date is the last day of the month df['Date'].dt.is_leap_year # Determine if it is a leap year df['Date'].dt.month_name() # Return the English name of the month df['Date'].dt.to_period('Q') # M for month, Q for quarter, A for annual, D for daily df['Date'].dt.weekday_name # return to the day of the week in English Due to pandas version problems, change pandas version in cmd type: pip install --upgrade pandas==0.25.3 () The # function converts the time in the given series of objects to midnight. '''
to this article on pandas filtering data by date range of the realization of the article is introduced to this, more related pandas date range filtering content please search for my previous posts or continue to browse the following related articles I hope you will support me in the future more!