cat bounce pool

I created cat bounce pool after seeing http://cat-bounce.com/ . In cat-bounce you can click a button to make it rain cats and I thought it’d be cool to increase the number of cats after it rains.  When I checked the source it was Flash, so I couldn’t see the code.  Time to make it in javascript.

I remembered a piece by mr. doob with bouncing balls, so I took that code and replaced the ball SVG with an SVG of a cat.  So cat bounce + ball pool = cat bounce pool. It’s a really simple hack, but I love how easy it can be to make these mash-ups when the code is open.

You can check out the code for cat bounce pool on Github, and here you can play with cat bounce pool. Click around to add cats.  Shake the browser to send them bouncing around off the sides.

programmatically liking things on facebook

Continuing along the lines of using social media as a medium, I’ve been playing around with facebook’s API recently and wanted to “Like” things.  I couldn’t find how to do it (more on that later) so I started to play with client side ways to do it. You know, because sometimes you’re just way pumped and just want to “Like” everything.

Here is a script that collects all of the “Like” buttons for comments and items currently loaded in the browser and clicks them.

 

I used it a couple times and I got this message:

So it’s probably best to include some time delay. I’m not really sure what the acceptable rate of liking is though. Basically if you use it for say <50 things in a second its fine. I did this a lot while writing and testing it. It's when I scrolled down to load a ton of stuff- and then used it- that I got the message. I'd test it again, but I'd rather not have the power to 'like' revoked. If you figure this out, do share!

Also, it turns out you can Like things with the Facebook API. You just find the ID of the item, and then send a POST request to https://graph.facebook.com/ID/likes.
This requires extended permissions to publish actions.

writing a web scraper over HTTP with nodejs

While working on a javascript web app I thought it would be pretty cool to write an API that I supply a URL and it would send back the data I want to scrape from the URL.

The basic flow here is that I encode the URL and include it as part of a get request, and then on the server side I retrieve the page source and scrape from it, sending back the data I want.

In this particular case I have Instagram URLs, and I want to scrape the location of the image.  Here is a quick rundown.

I have this URL for example: http://instagr.am/p/L9RF3SxC9H/

I encode the URL with encodeURIComponent and get “http%3A%2F%2Finstagr.am%2Fp%2FL9RF3SxC9H%2F”

I use that in a GET request to call a node script I wrote on the server.  This script fetches the source of the image and sends it back, in this case http://distilleryimage7.instagram.com/8011e852b81f11e1a8761231381b4856_7.jpg

Note: While writing this post I found a quick way to do this particular case with Instagram without having to scrape the page for its source.  You just append the URL with ‘media/?size=l’, so the above becomes ‘http://instagr.am/p/L9RF3SxC9H/media/?size=l’.  You can use that as your img src and it will render the image.  I’ll continue with the post just as an example to represent the idea, you can adjust the script- changing the jQuery selectors- to pull data specific to the page you want to scrape from.

I’ll just include the route that refers to the scraping.  In this I include the packages express, jsdom, util, request, and fs.  You can install them with npm.

I referred to this awesome post and associated code on using jQuery selectors on the server side to pull particular parts of the page source.

Here is the node js code

In the above, the ‘link’ variable in the http request is the encoded url. I retrieve that link from the express req object, req.params.link, then I used request to fetch the source from that URL, and then use jsdom and jQuery to pull the specific data I want.

Some close up screenshots of real time geolocated tweets

I built a tool yesterday to observe geolocated tweets as they came in on a world map.  Its kind of cool to zoom in on specific cities.  After just 30 seconds I gathered quite a few in New York.  Clicking on a marker reveals the profile pic and the tweet.

And then zooming out I could see clusters along cities in the Northeast.

After several minutes I zoomed out to see the world map, but at that point there were too many tweets and my browser crashed.  This is where selective coarsening would be useful.

You can find the project on github.

Using nodejs event streams and leafletjs to display geotagged tweets on a world map

I thought it would be cool to see tweets come in live using the Twitter Streaming API querying with location.  When there is a location bounding box, all tweets that come in have geocoordinates (though a small fraction are null).

Initially I wanted to focus in on a city- Leaflet maps look incredible when you zoom into a city- but the Twitter Streaming API was taking too long to fetch tweets while testing.  I set the bounding box to the world.  You can change the twitter fetch bounding box, as well as the initial mapping bounding box.

This is my first time using them, but from what I understand nodejs event streams allow you to send chunks of data to the browser from the server, as they come in.  This is pretty cool for real time applications.  I wanted to focus this application on immediate tweets, and right now there is no database.  Whenever you run it, you get whatever is coming in the twitter pipeline.

Take a look at the project here.

UPDATE: Note that you need to edit the config.js file with a twitter name and password because querying the streaming API with location is otherwise forbidden.  If you are using an account that already has many apps that query the API constantly (tweet harvesting for example), then you may experience a lag in rate of fetching.  This should not be an issue for most people and can be easily remedied by creating another twitter account to query the API with.

A quick exercise web scraping with javascript and bash in bandcamp

Here is a quick exercise to explore web scraping in bash on Bandcamp.  We’ll get our hands dirty with some simple console and bash scripting. This is written in January 2012, so things may change between now and when you read this.

example album page

If you take a look at the source on Bandcamp, you notice that all of the music data is located in a Javascript variable.  On your browser there should be a ‘view-source’ option.  In Chrome on the mac it is command-option-u.

 

Notice that in the comment Bandcamp includes information on their terms of service and their stance on applications that pull music off the site. You should probably read those and remember this is only meant as an exercise.  Looking at the FAQ I imagine they would be okay with this tutorial.  (Otherwise, if not okay, message me and I’ll take this down.)

So open up your web console.  I’m using Chrome, so it is command-option-i.  You can then explore the object TralbumData. You can retrieve the track information with TralbumData[‘trackinfo’].

We’re going to pull the titles and the files.

First get the titles.  You can use `console.log(str) ` to get things printed onto the console. We’ll write a loop over all of the objects and print out the ‘title’ value for each.

for (i in TralbumData[‘trackinfo’]){console.log(TralbumData[‘trackinfo’][i][‘title’]);}

Now copy that and paste it in a txt file called titles.

Then we need to get the URLs for the files. We can use the same loop, replacing ‘title’ with ‘file’

for (i in TralbumData[‘trackinfo’]){console.log(TralbumData[‘trackinfo’][i][‘file’]);}
 That will give you a list of URLs.  Put that in a file called playlist. Now in Bash we’ll download the file at each URL to the appropriate title.mp3 file. It took some googling to figure this out since I don’t normally do any bash scripting beyond simple command line tools.  Open up Terminal.  First we’ll create an array called name and put each line of titles to an entry in name. In doing this we also have the change the Internal Field Separator.  This is so that when we loop over cat titles we’ll get a new item for each line, rather than for each word.  You can do that with this command:
IFS=’
The second quote is on a new line define the IFS as newlines (IFS=’\n’ doesn’t work [source]). And then run the loop to fill the names array.

j=0;for i in `cat titles`; do names[$j]=$i; j=$(expr $j + 1); done

names contains the track titles.  To get the value out for index 0, you’d use ${name[0]}.  We’ll use that for the output file name.  This time we’ll loop over the entries in playlist and use wget to download the file at each url.

j=0;for i in `cat playlist`; do wget -O ${names[$j]}\.mp3 $i; j=$(expr $j + 1); done

The -O command line switch for wget specifies that the following argument will be the name of the output file.

 

text+icons

A way to explore the visual language of icons through messages and poetry

text+icons is a way to play with the icons of the Noun Project. You can type a message, and any words that match icons will be displayed as the icon. So you get a message thats a mix of text and icons.

I find the Noun Project pretty exciting – an intuitive visual language that can span many spoken languages. I like the idea of developing tools that will use the language, and can in turn potentially help the language grow. text+icons is exploring that idea.

The URL updates to reflect your message. You can write a little icon poem and send the URL to friends and they can see your text+icon creation when they open it in their browser. Here is one by james that I like:

my coffee addicted surfing robot sloth drives a jeep and loves physical-therapy and golf

and the accompanying URL: http://texticons.whichlight.com/#my&coffee&addicted&surfing&robot&sloth&drives&a&jeep&and&loves&physical-therapy&and&golf&

Shortened: http://bit.ly/oNEiB4

You can share the message via twitter too, using the share buttons below the input box. The URL is automatically shortened using bit.ly.

To make this, I used bottle to retrieve the icons and javascript to visualize it all. Bottle is running using Passenger. You can see the source and remix the project here.  There is also a pretty simple API to the noun project icons that I wrote.  At the time this was made, I could not find one, but there will probably be one presented in the coming months by the Noun Project.

You can get an icon of a particular noun as an img or a json.

For the json:
just search the [noun] with http://texticons.whichlight.com/api/[noun], and you’ll get back a json with {‘status’ : ‘ … ‘, ‘data’ : ‘…’} where status will be ‘Success’ if there is an icon, and ‘Fail’ if there is not. ‘data’ will return back the icon link if there is an icon. It just retrieves the first icon if there are multiple ones.

For the img:
just search the [noun] with http://texticons.whichlight.com/icon/[noun] and you’ll get an img.

COOL!