Posts

Hacker News Dataset October 2016

Image
Our latest project on Sizzle  is a visualization of the Top 10k Posts of All Time  on Hacker News . To create the visualization, we first needed to collect the data. I noticed that there was an old copy of the hacker news dataset available on Big Query . But I needed an up-to-date copy, so I looked into the Hacker News Firebase API . The API allows you to get each item by Id. You can start by retrieving the current Max ID, then walking backwards from there. (Items my be stories, comments, etc., it's the same API endpoints for all types of items.) There is no rate limit, so I created the following script that will generate a text file with 10MM lines containing all of the URIs to retrieve. (we will then feed this file into wget using xargs) Note: 10MM items was ~5 years worth of data. Script to create the 10MM line file of URIs to retrieve: https://gist.github.com/aaronhoffman/1f753c660d7364bb594a36af350b227c That script takes about 10 minutes to produce a file t

Top Cities in the United States

To find a data source for visualizations , I found myself commonly searching for "Top 200 Cities in the US", "largest cities by land area", and "largest meto area by population", etc. I would then have to combine the list I found with other data sources to get all the info I needed. I thought I'd try to keep an up-to-date copy of this list, in case others also found this helpful. You can find that list here:  https://gist.github.com/aaronhoffman/e1893d32fa1254429abf57f5c0413fa3 I will try to keep this list up-to-date with additional information and API keys as I use it over time. Hope this helps, Aaron

Reddit API OAuth C# .NET Example

Image
I had a need to connect to the Reddit API via C# .NET, and thought I'd put this little tutorial together since most I found were in java or python. Note: General Reddit OAuth API info:  https://www.reddit.com/dev/api/oauth 1. Go here and make an app:  https://www.reddit.com/prefs/apps 2. Use the values from the app you created above, in this code sample:   gist:  https://gist.github.com/aaronhoffman/b59585d507601b05d8db02493eaaf73e   more info:  https://github.com/reddit/reddit/wiki/OAuth2 3. Make API requests:   gist:  https://gist.github.com/aaronhoffman/b59585d507601b05d8db02493eaaf73e Check out the result here : interactive data visualization Hope this helps, Aaron

VirtualBox Right-Click Windows Taskbar On Mac

Using VirtualBox on my Mac (in windowed mode, not full screen), I noticed that right-click on the Windows taskbar was not working. I commonly use right-click on the task bar to launch an application as administrator, pin an application to the taskbar, etc. I could two-finger click on other windows to get the right-click menu to appear, but that did not work on the taskbar. To get the right-click menu to appear on the windows taskbar, I had to hold down the "host key" which for me was the left Mac command key, and single-finger click the item in the taskbar. Hope this helps! Aaron

ASP.NET MVC 5 User Admin

AKA (for google-fu): asp.net mvc 5 web site administration tool asp.net mvc 5 web configuration tool asp.net mvc 5 identity asp.net mvc 5 membership I have missed the asp.net Web Site Administration Tool that used to be included with Visual Studio. I created an alternative for asp.net mvc 4 you can find here . For asp.net mvc 5, I created an  Azure Site Extension  to provide the same functionality. It can be run from VS on your local machine as well as installed as a site extension on Azure. Links: extension:  https://www.siteextensions.net/packages/AspNetUserMaintenanceAzureSiteExtension/ github repo:  https://github.com/Stonefinch/AspNetUserMaintenanceAzureSiteExtension Hope this helps, Aaron UPDATE These are now published to nuget.org  https://www.nuget.org/packages/AspNetUserMaintenanceAzureSiteExtension/

Gather Metadata For Each Column of SQL Table

Image
Often when working with a new data set I'll run these queries to learn more about each column of data. https://gist.github.com/aaronhoffman/eb30805ee2f5cafc64152dd1def800bd For example, you can run a single query to union together aggregate data on each numeric column. Result looks something like this: Hope this helps, Aaron

Get Line Numbers In Exception Stack Trace

Image
If you want to ensure you get line numbers in your Exception Stack Trace, you need to make sure your project is set up to provide "full" debug info on build. By default, for "Release" configuration, this is set to "pdb-only". Steps to enable: In Visual Studio... 1. Ensure your project is set to "Release" Solution Configuration (or whichever config you use for deployments) 2. Go to Project > Properties > Build 3. Click the Advanced Button in the bottom right. 4. Under Output > "Debug Info", choose "full" from the dropdown. 5. Save all settings and build your project. You will now see line numbers in Exception Stack Traces. Hope this helps, Aaron