Efficiently Convert Timezones In Pandas Dataframe

I have a large pandas dataframe (tens of millions of rows) which includes a column for UTC time and the time zone. I want to create a column which contains the local time for the r

Solution 1:

Demo:

Source DF:

In [11]:dfOut[11]:datetimetime_zone02016-09-19 01:29:13    America/Bogota12016-09-19 02:16:04  America/New_York22016-09-19 01:57:54      Africa/Cairo32016-09-19 11:00:00    America/Bogota42016-09-19 12:00:00  America/New_York52016-09-19 13:00:00      Africa/Cairo

Answer :

In [12]:df['new']=df.groupby('time_zone')['datetime']\.transform(lambdax:x.dt.tz_localize(x.name))In [13]:dfOut[13]:datetimetime_zonenew02016-09-19 01:29:13    America/Bogota2016-09-19 06:29:1312016-09-19 02:16:04  America/New_York2016-09-19 06:16:0422016-09-19 01:57:54      Africa/Cairo2016-09-18 23:57:5432016-09-19 11:00:00    America/Bogota2016-09-19 16:00:0042016-09-19 12:00:00  America/New_York2016-09-19 16:00:0052016-09-19 13:00:00      Africa/Cairo2016-09-19 11:00:00