Besides not being very transparent about the algorithms most of the time, they create a currency you need to trust based on a gut feeling, but that’s another story.

A good trend is more and more governments and institutions are opening up their datasets to let the public access their data. Whether it is because a governments is legally obliged or a company/institute believes it can benefit from opening up their sources, we have access to some data to play with.

I’ve been reading up lately on the subject of data, and I went to resonate.io where I was inspired. @flowingdata also compiled a nice list of blogs/sites on the subject on his site.

Working with visual data has always been of interest:

Armed with a Silicon Graphics Indigo 2 Extreme desktop workstation with R4400 processor, four gigabyte hard drive, and 128 megabytes of RAM, Borchers uses IRIS Explorer an interactive 3D data visualization system to analyze data. The team’s datasets range in size from four to 20 megabytes (based on foot scans producing 300,000 points in x-y-z spatial datasets). 1999 web 1.0 link

Now that I have 128Mb of ram I thought I should give it a go. I chose processing as I used it before for different tasks and it’s is a good protoyping tool used by dataviz pro’s to make stuff like cascade (built by @blprnt  at @nytlabs). Besides that, I recently had a nice re-intro to processing  by @vormplus and I followed a workshop @p5Ghent.

If you’re interested here are some other tools as well

Having just missed the Ghent appsforghent challenge it felt a bit odd at first to use the data.gent.be sets, but I’m a local, and you should think local(but act global, which is why this post in in English).

The Idea

An interactive explorative tool that allows you to visualise data on a map (of (part of) Ghent)

I wanted to express the fact that this particular place where I live is crowded, densely populated and this is why I started thinking about using physics. Particles bumping into each other, fighting for space.

This is far from finished but I thought I’d share this already.

It’s on github, next up is giving kids more space by mapping the demographic pyramids on this data and making the particles representing kids bigger. With the press of a button I could show what impact building a new high-rise flat will have in a certain area.

The user could set the desired sq meters of outdoor area per capita and see which neighborhoods are comfy and which are crowded. Much more thing a possible, but also a little more time is needed.

Right now it reads in map data from an SVG file, converts those shapes to box2D entities and populates them with particles.

the small points mean < 1 person!

The screenshot isn’t using the correct data yet but you don’t need a lot of imagination to see that it has more potential then the original

just using a map to display map related is a start and while that is an option on the site (which is a great source of info btw, well presented and all that) this static mapping doesn’t really say a lot. It’s crowded. period.

The exploration

I’m just using this example to explore the possibilities of using physics for making sense to data. This doesn’t necessarily mean you need to map phyisical properties to your data that in real life exhibits the same properties. I can imagine  that if you need to sort certain types of data it could make sense to let them have a different density so some data floats while other would sink, while you’re actually visualizing eg. twitter vs. facebook.

to be continued

To avoid repetetivity, I chose to seed the algorithm with somewhat random data in the form of a series of pixels extracted along the image of the earth.

I work at a hi-tech internet  innovation research institute but are in no way affiliated to Silicon Valley, we came up with the idea of connecting Ghent with a line to silicon valley, the result, is not that interesting to show, but it is a line, one pixel wide. It is however an interesting series of pixels, where neighbouring pixels are related.

but what to  to with pixels.

It’s not an awful lot of data. A color, an RGB value(3 times 8 bits), a sequence number perhaps, maybe the relation to the neighbor? Does it within a certain threshold have the same amount of blue, green and red? Or does it represent a bump? As it comes from a terrain map, maybe  it can represent an altitude. not sure yet.

But what I was looking for was more something that contains enough data to allow to generate specific images

You’re looking at it, just above this sentence. after parsing the pixel color data from it, half an alien is generated according to this procedure.

which is then mirrored and completed. Repeat this for every pixel and you get a lot of different aliens, first 25 of the above line, pictured below

[processing code]

/**
* aliens
*/

PImage lijn;
int px = 0;
int pxmax = 5;
int offsetY = 0;
int alienSize = 160;

void setup() {
frameRate(4);
size(pxmax*alienSize, pxmax*alienSize);
smooth();
background(0);
noStroke();
lijn = loadImage(“lijn.png”);

}
void alien(color code, int px){
//println(red(code));
//int Rr = Integer
int offsetX = px – (pxmax*offsetY) ;

fill(red(lijn.pixels[px]),green(lijn.pixels[px]),blue(lijn.pixels[px]));
rect((offsetX*alienSize),(offsetY*alienSize),alienSize, alienSize);
fill(0);

String alienR = Integer.toBinaryString(parseInt(red(code)));
alienR = “0000000″ + alienR;
alienR = alienR.substring(alienR.length()-8);
String alienG = Integer.toBinaryString(parseInt(green(code)));
alienG = “0000000″ + alienG;
alienG = alienG.substring(alienG.length()-8);
String alienB = Integer.toBinaryString(parseInt(blue(code)));
alienB = “0000000″ + alienB;
alienB = alienB.substring(alienB.length()-8);
//
//println(alienR);
for(int i=0;i<8;i++){
//println(alienR.charAt(i));
}

for (int col = 0 ; col < 4; col++) {
for (int row = 0; row < 8; row++){
if(col == 1) {
String[] alienRArray = alienR.split(“”);
if(parseInt(alienRArray[row+1]) == 1) {
rect((col*20)+offsetX*alienSize, (row*20)+offsetY*alienSize, 20, 20);
rect(((7-col)*20)+offsetX*alienSize, (row*20)+offsetY*alienSize,20,20);
}
}
if(col == 2) {
String[] alienGArray = alienG.split(“”);
if(parseInt(alienGArray[row+1]) == 1) {
rect((col*20)+offsetX*alienSize, (row*20)+offsetY*alienSize, 20, 20);
rect(((7-col)*20)+offsetX*alienSize, (row*20)+offsetY*alienSize,20,20);
}
}
if(col == 3) {
String[] alienBArray = alienB.split(“”);
if(parseInt(alienBArray[row+1]) == 1) {
rect((col*20)+offsetX*alienSize, (row*20)+offsetY*alienSize, 20, 20);
rect(((7-col)*20)+offsetX*alienSize, (row*20)+offsetY*alienSize,20,20);
}
}
}
}
if(offsetX == 4){
offsetY++;
}
}
void draw(){
lijn.loadPixels();
color code = color(red(lijn.pixels[px]),green(lijn.pixels[px]),blue(lijn.pixels[px])) ;
alien(code, px);
px++;
}

[/processing code]

Of all the cruel ways we came up with to punish spammers, or all the gentle ways of changing their behavior, there was one that seemed spot on. As a plus it was easy to implement, needed because the show ends in just two days.

1. find spammer
2. I think  you noticed the spam, just click it to open the spammers profile
   

   Notice the two red buttons, one is remove user, the other is mark as spammer

   

   spam mainly comes from uneducated kiddies, not viagra sellers as there is bot protection

3. mark the user as spammer, a websocket message is sent to the user forcing a reload, and making the villachat…
   

   nyanchat, replacing the spammers(mine in this screenshot) text input window with nyancat

4. The user could still log out and create a new account but their logout (afmelden) button is gone, there session stays open and they can’t create a new username.
   

   regular users can logout

   

   spammers can't sign out, button is gone

5. thanks nyancat
6. As a bonus the nyansound is there too and the session stays signed in for four days. That should teach em. Oh yeah talking about teaching clicking on the cat opens the wikipediapage on spam, educating the user.

Feel free to check out villa.een.be, but don’t make me nyan you

Thanks to everyone who helped thinking up this and other ways to punish/educate spammers. Happy spamming,I’m coming to get ya!

on the kaotec side: the software used for the CA-tracks is homemade and on the release. pure data and probably unstable… go grab it and enjoy

Enjoy