WWDC Analysis 2

We are now going to do a time series analysis of Twitter data during Apple's WWDC conference. We will analyze all #wwdc tweets first, and then separate the tweets into Apple's four main software prodcuts: iOS, macOS, watchOS, and tvOS. Lastly, we will do a time series analysis of the rumored but missing new macbook pro hardware. This was rumored to announced at WWDC and wasn't. We will show that some Apple fans were a little unhappy with the lack of new hardware.

Let's begin by loading our genuine/organic tweets from our first analysis.

#Load organics pkl file
organics = pd.read_pickle("organics.pkl")
print organics.count()

Here is the output:

_id                65018
coordinates          127
created_at         65018
followers_count    65018
friends_count      65018
hashtags           65018
screen_name        65018
source             65018
text               65018
dtype: int64

We are now going to put our tweets into 1-minute bins and count the number of tweets in each bin. That is, how many tweets per minute.

#To look at the number of tweets per hour, we have to resample our tweet collection. Everyting is in UTC time.
ts_hist = organics['created_at'].resample('1t').count()
ts_hist.head()

Here is the output:

created_at
2016-06-13 16:45:00    144
2016-06-13 16:46:00     94
2016-06-13 16:47:00    120
2016-06-13 16:48:00    116
2016-06-13 16:49:00    122
Freq: T, dtype: int64

Here we can see the first few minutes of recording tweets. I started recording 15 minutes before WWDC started and stopped recording 15 minutes after WWDC ended.

#Time series plot. Let's make it big because I want to discuss some of its peaks.
ts_hist.plot(color='red',figsize=(14, 10))

The following is a visualization of the number of tweets per minute during WWDC. Remember everything is in UTC time.

You can literally see the number tweets rapidly increase when WWDC starts at exactly 17:00 UTC and drop as it ends at 19:00 UTC. Notice the sharp jump in the 15 minutes after WWDC ends as people put in their "final thoughts." See that huge spike at 18:30? That's exactly when apple anounced its changes to the iPhone messages app including bubble effects, handwritten messages, and full screen effects like fireworks. Twitter went crazy apparently. See the large spike at around 17:45 UTC? That's exactly when apple announced Siri would be available on MacOS Sierra, its next operating system for macs. In summary, every major announcement comes with a spike, as it should!

We are now going to do a comparison of time series for iOS, macOS, watchOS, and tvOS. That is, we're going to attach as much data as possible to each of these four terms and do a time series analysis.


#We need to make four new pandas dataframes from organics for ios, macos, watchos, tvos.
my_ios_organics = organics[organics['text'].str.contains('ios', case=False)==True]
my_10_organics = organics[organics['text'].str.contains('10', case=False)==True]
my_iphone_organics = organics[organics['text'].str.contains('iphone', case=False)==True]
ios_organics = pd.concat([my_ios_organics,my_10_organics,my_iphone_organics],ignore_index=False)

my_watch_organics = organics[organics['text'].str.contains('watch', case=False)==True]
my_watchos_organics = organics[organics['text'].str.contains('watchos', case=False)==True]
my_3_organics = organics[organics['text'].str.contains('3', case=False)==True]
watchos_organics = pd.concat([my_watch_organics,my_watchos_organics,my_3_organics],ignore_index=False)

my_mac_organics = organics[organics['text'].str.contains('mac', case=False)==True]
my_macos_organics = organics[organics['text'].str.contains('macos', case=False)==True]
my_sierra_organics = organics[organics['text'].str.contains('sierra', case=False)==True]
macos_organics = pd.concat([my_mac_organics,my_macos_organics,my_sierra_organics],ignore_index=False)

my_tv_organics = organics[organics['text'].str.contains('tv', case=False)==True]
my_tvos_organics = organics[organics['text'].str.contains('tvos', case=False)==True]
tvos_organics = pd.concat([my_tv_organics,my_tvos_organics],ignore_index=False)

#One more dataframe for analysis on missing macbook pros later
macbook_organics = organics[organics['text'].str.contains('macbook', case=False)==True]

#Let's do the resampling for ios, watchos, tvos, and macos.
ios_hist = ios_organics['created_at'].resample('1t').count()
watchos_hist = watchos_organics['created_at'].resample('1t').count()
macos_hist = macos_organics['created_at'].resample('1t').count()
tvos_hist = tvos_organics['created_at'].resample('1t').count()

os_total_hist = DataFrame({'ios' : ios_hist, 'watchos' : watchos_hist, 'macos' : macos_hist, 'tvos' : tvos_hist})

#Time series plot. Let's make it big.
os_total_hist.plot(figsize=(14, 10))

The following is a visualization of the number of tweets per minute corresponding to iOS, macOS, wathcOS, and tvOS.

This time series agrees with the WWDC program as expected. With this visualization, you can literally recreate how Apple organized its time to discuss each of its four main software platforms. Pretty cool in my opinion.

I'm now going to turn to the issue of the highly anticipated, but missing macbook pros at WWDC.
First are tweets before WWDC starts.

#Tweets containing the word macbook before WWDC starts
print macbook_organics['text'].head(n=25)

Here is the output:

created_at
2016-06-13 16:45:17     praying for a new MacBook 13 inch šŸ˜‚šŸ˜‚ #WWDC2016
2016-06-13 16:45:19    Keynote key at #WWDC2016 : #swiftlang , #Siri ...
2016-06-13 16:45:35    Officailly excited for #WWDC2016 as a macbook ...
2016-06-13 16:45:53    #WWDC2016 predictions: macOS, excessive boredo...
2016-06-13 16:46:48    Answer my Macbook Pro prayer, Tim! #WWDC2016 h...
2016-06-13 16:47:12    YET TO START\n\n#WWDC2016 kicks starts today!\...
2016-06-13 16:47:13    I wonder when Apple will finally announce a Ma...
2016-06-13 16:48:15    All I want from #WWDC2016 is a new MacBook Pro...
2016-06-13 16:48:19    #News: Live coverage of WWDC 2016: iOS 10, mac...
2016-06-13 16:48:32    @tim_cook Long overdue. Please announce a new ...
2016-06-13 16:49:30    The only reason yo have #Safari on my MacBook ...
2016-06-13 16:49:39    What are you expecting on the stage of WWDC16?...
2016-06-13 16:49:47    #WWDC2016 expectations:\nmacOS, iOS 10, watchO...
2016-06-13 16:49:49    ok, ok i had mad about  #Google but about to w...
2016-06-13 16:50:10    ok, ok i am mad about  #Google but about to wa...
2016-06-13 16:50:12    The only reason to have #Safari on my MacBook ...
2016-06-13 16:50:28    Keeping my eye on Apple's WWDC16 conference to...
2016-06-13 16:51:06    Forewarning that I will throw another temper t...
2016-06-13 16:51:07    Also, if the MacBook Pro's get a refresh today...
2016-06-13 16:51:59             macbook pro 2016 better be lit #WWDC2016
2016-06-13 16:52:16    ā˜‘ļø Cup of tea\nā˜‘ļø Chocolate digestives \nā˜‘ļø So...
2016-06-13 16:53:23    So, is there going to be a new MacBook Pro? #W...
2016-06-13 16:53:30                     Please new Macbook Pro #WWDC2016
2016-06-13 16:53:40    I'm ready #wwdc16 #apple #keynote #ipad #iphon...
2016-06-13 16:53:41    #WWDC16 I'm excited! Are you? Let's hope for a...
Name: text, dtype: object

Tweets after WWDC ended.

#Tweets containing the word macbook after WWDC ends
print macbook_organics['text'].tail(n=25)

Here is the output:

created_at
2016-06-13 19:12:40    No new hardware at #WWDC2016, will have to wai...
2016-06-13 19:12:51    #WWDC2016 where is the New MacBook Pro? Thanks...
2016-06-13 19:12:56    We already have PayPal, a clock on our iPhone,...
2016-06-13 19:13:18    Super pissed off that no word or update on the...
2016-06-13 19:13:26    12:40 p.m just awake to see how good MacBook P...
2016-06-13 19:13:27    #WWDC2016 no new products? #iPhone #macbook #imac
2016-06-13 19:13:36    No new MacBook pro or iPhone announcements tod...
2016-06-13 19:13:39    Super bummed that new MacBook pros weren't ann...
2016-06-13 19:13:39    C'mon @Apple No new #MacBookPro?? Gotta wait t...
2016-06-13 19:13:48    Told my dad to wait until mid June to buy new ...
2016-06-13 19:13:50    @rahulgdsouza @gopalvirtual #macbook2016 It wa...
2016-06-13 19:13:53    Time to replace my MacBook Pro 2009 #WWDC2016 ...
2016-06-13 19:13:54    I've held off for years for new MBP; don't wan...
2016-06-13 19:14:02    So no #MacBookPro?dafuq?pretty sure that if I ...
2016-06-13 19:14:15    That was a bit pants , we came for MacBooks an...
2016-06-13 19:14:19    So iOS and macOS gets up to date with 2016, bu...
2016-06-13 19:14:29    No new #MacBook Pro during #WWDC2016 so I swit...
2016-06-13 19:14:50    Tune in for the new MacBooks, leave with the s...
2016-06-13 19:14:53    Apple announced exactly what was expected and ...
2016-06-13 19:15:00    I'm sad there was no MacBook Pro mentioned tod...
Name: text, dtype: object

Look how hopeful everyone is about new macbook pros 15 minutes before WWDC, and look how disappointed they are afterwards.

We're going to do one more time series visualization corresponding to the term "macbook."

#Let's take a look at a time series for the term macbook. 
macbook_hist = macbook_organics['created_at'].resample('1t').count()
macbook_hist.plot(color='red',figsize=(14, 10))

Notice the overlay with the purple MacOS times-series graph above. And, of course, notice the explosion after WWDC ended when people realized there were no new macbook pro computers announced. Really beautiful graph in my opinion.