Showing posts from October, 2016

Hacker News Dataset October 2016

Our latest project on Sizzle  is a visualization of the Top 10k Posts of All Time  on Hacker News . To create the visualization, we first needed to collect the data. I noticed that there was an old copy of the hacker news dataset available on Big Query . But I needed an up-to-date copy, so I looked into the Hacker News Firebase API . The API allows you to get each item by Id. You can start by retrieving the current Max ID, then walking backwards from there. (Items my be stories, comments, etc., it's the same API endpoints for all types of items.) There is no rate limit, so I created the following script that will generate a text file with 10MM lines containing all of the URIs to retrieve. (we will then feed this file into wget using xargs) Note: 10MM items was ~5 years worth of data. Script to create the 10MM line file of URIs to retrieve: That script takes about 10 minutes to produce a file t

Top Cities in the United States

To find a data source for visualizations , I found myself commonly searching for "Top 200 Cities in the US", "largest cities by land area", and "largest meto area by population", etc. I would then have to combine the list I found with other data sources to get all the info I needed. I thought I'd try to keep an up-to-date copy of this list, in case others also found this helpful. You can find that list here: I will try to keep this list up-to-date with additional information and API keys as I use it over time. Hope this helps, Aaron

Reddit API OAuth C# .NET Example

I had a need to connect to the Reddit API via C# .NET, and thought I'd put this little tutorial together since most I found were in java or python. Note: General Reddit OAuth API info: 1. Go here and make an app: 2. Use the values from the app you created above, in this code sample:   gist:   more info: 3. Make API requests:   gist: Check out the result here : interactive data visualization Hope this helps, Aaron

VirtualBox Right-Click Windows Taskbar On Mac

Using VirtualBox on my Mac (in windowed mode, not full screen), I noticed that right-click on the Windows taskbar was not working. I commonly use right-click on the task bar to launch an application as administrator, pin an application to the taskbar, etc. I could two-finger click on other windows to get the right-click menu to appear, but that did not work on the taskbar. To get the right-click menu to appear on the windows taskbar, I had to hold down the "host key" which for me was the left Mac command key, and single-finger click the item in the taskbar. Hope this helps! Aaron