4 ms·

Show HN: Scottish Independence Referendum Heatmap

30 points by chriswait 12y ago

chriswait 12y ago
The (Scottish) startup I'm working for builds a contextual publishing platform called Bubbal ("context" based on a user's location and interests, as well as time of day). We're using it to collect and heatmap yes/no opinions, worldwide, for the Scottish Independence Referendum next month. We'd love to know how you feel, and I'm particularly interested in any feedback you can give me on performance or UX.
- tlarkworthy 12y ago
  The kernel size (smoothing width) should be chosen based on something like leave-on-out cross validation. At the moment the sampling is too sparse and everything is independent and not smooth at all. This is described in the intro of "Assessment of probability density estimation methods: Parzen window and Finite Gaussian Mixtures" http://www0.cs.ucl.ac.uk/staff/c.archambeau/publ/iscas_aa06.pdf http://www0.cs.ucl.ac.uk/staff/c.archambeau/publ/iscas_aa06.... (as well as many other places that just happened to be the first google result) email me if you want more details, I am ex-informatics.
  - chriswait 12y ago
    This is massively useful, I was looking for a way to 'aggregate' results somewhat. I will read through this today and might send you a question or two later on if you don't mind.
    - tlarkworthy 12y ago
      Yeah send any queries over or I can go through it face to face if you want.
- wambotron 12y ago
  This is interesting, but I feel like I have no real knowledge of the situation to give a good vote. I voted 'yes' as I assumed, after looking at the map, that most Scots want it. It doesn't matter to me, though. I'm not Scottish!
  - smcl 12y ago
    Interestingly "Scottish"ness doesn't necessarily matter for eligibility to vote in this referendum. I was born and raised there for 25 years but cannot vote as I'm not resident there. Those who live in Scotland can vote, regardless* of their nationality or how they identify. Which is of course correct and reflects what I'd like an independent Scotland to look like - immigrant friendly and diverse. * = within certain restrictions, something like a couple of years resident prior to the referendum. Don't quote me on that :)
moron4hire 12y ago
I wanted to see what you have so far, but since I'm not Scottish, I didn't want to vote first. I denied access to location data, received a message saying I couldn't vote without location, and then had no way to view the current data on the map.
- chriswait 12y ago
  Thanks for the feedback! You make a good point: We're asking before we're giving anything. Completely understandable to deny location services. Would your preference be to see the heatmap worldwide before being asked for your location?
  - cwmma 12y ago
    or maybe just a shortcut to see results button
  - moron4hire 12y ago
    Well, if you show the heatmap right away, it might skew your results. Perhaps if you had a third option "skip voting and view results".
nly 12y ago
Looks like a population map.
- valarauca1 12y ago
  Because it basically is. XKCD even did a comic about this http://imgs.xkcd.com/comics/heatmap.png http://imgs.xkcd.com/comics/heatmap.png
- viraptor 12y ago
  When the colours are summed up? I would hope it is - otherwise data is missing. But you can definitely see differences between cities when you zoom up. For example Glasgow seems to be a primarily "No" place, but Kirkcaldy is mostly for "Yes".
GordonS 12y ago
Doesn't work for devices that can't accurately provide location information. For example, I live near Aberdeen, but my ISP is in London - so this shows me as in London. Doesn't seem to be a way to override the 'detected' location?
- chriswait 12y ago
  You're right - we haven't done this yet as we anticipated most of the traffic to come from mobile devices. I will have a look an manually entering location today (now wondering if an address search box is better, or "drop a pin"?).
  - jarofgreen 12y ago
    The location services on my phone are often off, or I may deny location permission until I see more what the site is. Bad assumption. (Good luck with project tho!)
maxbog 12y ago
I think you are publishing locations that are too specific. I can identify houses, from which the votes were cast. This information can be abused and is therefore dangerous. Please do not publish such information without informing the user loudly and clearly of the implications of voting.
- chriswait 12y ago
  I think you're absolutely right. I was considering adding some noise to the lat longs collected to mitigate this. We're looking at the copy (to better highlight what data is stored and displayed). Thank you for the feedback.
  - Nowaker 12y ago
    Maybe place all the marks in the town center?
    - billybofh 12y ago
      That would diminish the usefulness of the map quite a bit. People will be interested in this on a ward-by-ward basis - I'm pretty sure. A small random 'shuggle' should be enough to help.
mrloop 12y ago
The colour blue for one of the result sets is a poor choice as the map is drawn in blue / turquoise. You need 2 colours for the results which are of a similar distinguishability against the map colour.
- chriswait 12y ago
  Good call, having a think about choosing other colours now. Thanks for taking a look!
chestnut-tree 12y ago
This is a nice idea. My UX suggestions: - display the heatmap first - have a prominent link saying something like: add your vote to the map. When someone clicks that link, ask them to enter their location (city or town and country), then ask them the same question the Scots will vote for in the referendum: Should Scotland be an independent country? Yes/No. I think the auto-detect location causes too many problems and may have privacy implications if, as another poster says, you can identify quite precise locations. What would be really nice would be a map similar to the Book Depository ("Watch people shop" map): http://www.bookdepository.com/live http://www.bookdepository.com/live In this case, you would display a pop-up each time someone voted with the name of their town or city and whether they'd voted Yes or No.
- chriswait 12y ago
  Hi, thanks for all the feedback. I love the bookdepository.com user experience with the live updates. We'll be working on this as we get closer to the date! Auto detection of location is proving to be a bit tricky - we're seeing a lot of votes clustered around internet exchanges in places.
- mbesto 12y ago
  > - display the heatmap first - have a prominent link saying something like: add your vote to the map. When someone clicks that link, ask them to enter their location (city or town and country), then ask them the same question the Scots will vote for in the referendum: Should Scotland be an independent country? Yes/No. One nitpick - don't show the heatmap first if you want correct poll results. This heavily biases the undecided people into "oh, everyone in Aberdeen says No...I guess I say No then too!"
smcl 12y ago
I'm afraid I can't see a thing, is it down due to the load coming from HN? Just a white screen with a sort of google-maps-ish compass in a box. I'm very keen to see something like this, with the referendum so close! edit: back up now
- chriswait 12y ago
  Really appreciate you taking the look. The site is still very much up, could you try reloading the page?
pmtarantino 12y ago
You should post it on /r/Scotland on Reddit!
- chriswait 12y ago
  I was planning to give it a day (give the server a rest), but somebody beat me to it: http://www.reddit.com/r/Scotland/comments/2ej5kk/scottish_independence_referendum_heatmap/ http://www.reddit.com/r/Scotland/comments/2ej5kk/scottish_in...
midhir 12y ago
Looks interesting, congrats on getting it shipped! A couple of things: 1) At any kind of scale, with smaller data-sets, the blue and red are overlapping in places to form purple and the user has to work out from the hue which is doing better. E.g. Edinburgh looks like it has a high blue content but I need to zoom in to see that really it's dominated by a massive blue blob on Grassmarket. 2) (Wonkish) The YES and NO buttons are stacked vertically with no margin. If the order isn't randomised already I'd consider doing that and separating the buttons. 3) I'm not from Scotland, but I did vote. If you're collecting that data also, why not show it? It'd be interesting for sure. We can often see Scotland across the water from our town, there are plenty of people here who consider themselves Scottish and others besides who have a dog in the fight. Looking forward to the 18th anyways. Yes or No, it'll be a hell of a party!
- chriswait 12y ago
  Hi there, thanks for the feedback! 1. I'm looking at the way google maps heatmaps lump results together at different zoom levels. If anyone has any experience with this, I'd love to hear about it. 2. This is exactly right. On the todo list now. 3. If you voted outside scotland (and the browser got a location), then it'll be displayed on the map in the place you voted from. Is this what you meant? Really appreciate all the comments.
  - midhir 12y ago
    Ah cool! Starting to see some colours over here now. Belfast voting Yes and Dublin voting No.. Now there's a turnup for the books.
k-mcgrady 12y ago
You broke my browser (Safari). I said yes to giving location and it located me - then asked me again, and again and again. The only way out was to allow it to remember my decision for 24 hours. Then the site told me it couldn't locate me despite having a pin on my house.
- chriswait 12y ago
  Ah this sucks, sorry about that. I think I've found the source of this and it should be sorted now. Thanks for taking a look!
  - k-mcgrady 12y ago
    Great thanks. I'll give it another try! Edit: Working now. I'd love to see a breakdown of votes (total yes/no and world yes/no vs scotland yes/no).
    - chriswait 12y ago
      Have since added global totals to the key at the top right, working on per-country totals over the next few days. Thanks again.
AndrewGreen 12y ago
To be clear, what is the question -- is it "Do you want independence", or is it "Do you want unity"?
- Marazan 12y ago
  Should Scotland be an independent country?
- chriswait 12y ago
  Hi Andrew. Thanks for the feedback, have updated the copy on the 'welcome' message to reflect this. (This message is only displayed to first-time users of the site.)
- smackfu 12y ago
  It seems like on the ground in Scotland, the sides are just "YES" and "NO" at this point.
- smcl 12y ago
  This is a good point but interestingly within Scotland (i.e. the target audience) the question doesn't need to be explicitly stated. People will have "Yes" (or "Aye") or "No" badges on their facebook profiles without any context. It's probably good that they added the question in though as I'm fascinated by the responses outside Scotland!
s_dev 12y ago
You should include the total respective votes with the key so the accuracy of the data can be compared to other polls. Maybe even a break down of the UK/ Ireland/ Europe/ Rest of World.
- chriswait 12y ago
  I love this idea. Will probably start with the global breakdown first. Thanks for taking a look for us!
pawn 12y ago
I don't know anything about Scottish politics, but as a generality, it looks like most larger cities vote Yes while the smaller towns have people voting No. Can anyone with insight explain why that'd be?
jarofgreen 12y ago
What is "vote every 24 hours and show your support" about?
touristtam 12y ago
I don't think a heatmap is the best representation for this kind of data: 1. if this is meant to show both votes, then a pie chart for each position (town/council) would be more effective 2. the blur is not clearly visible at all when zooming out completely 3. taking into account 'votes' from outside the country concerned is a bit confusing as pointed out in another comment, the rules for the referendum states that you have to be in Scotland to cast you vote. However, I had a look at the http://bubb.al/ http://bubb.al/ website and I can see how this could be used in a contained area (like an attraction park) or more generally if you could tie this up with a social media to show the mood of a user population in a wide area. On a side note, I am surprised you are not located near the Quarter Mile. ;)
- chriswait 12y ago
  Thanks for the feedback! Looking at each point individually: 1. We went with a map over a series of pie charts as we'd like to create something a bit more engaging for the user. 2. Agreed: The blur is causing me several issues, and is something I'm absolutely still working on. 3. Obviously these "votes" won't be counted, we're just trying to get a sense of how people in different part of the world feel about it. Thanks for taking a look at our site! Attraction parks are a use-case we're absolutely something we're looking into. We're currently based in a tech incubator called CodeBase (http://www.thisiscodebase.com/ http://www.thisiscodebase.com/). Quarter Mile is a beautiful building, but you can't beat being surrounded by other startups!
touristtam 12y ago
Just a side note. The map is continuously trying to locate me in Birmingham which I guess is my ISP's end-point connection to the internet, whereas I am actually in Edinburgh. I am using a few different extension in chrome and in firefox that normally block advert/analytics request, but this time the extensions are disabled to check for interferences. So how to you get the location? Because it isn't working for me.

30 points by chriswait 12y ago

The (Scottish) startup I'm working for builds a contextual publishing platform called Bubbal ("context" based on a user's location and interests, as well as time of day). We're using it to collect and heatmap yes/no opinions, worldwide, for the Scottish Independence Referendum next month. We'd love to know how you feel, and I'm particularly interested in any feedback you can give me on performance or UX.

tlarkworthy 12y ago
The kernel size (smoothing width) should be chosen based on something like leave-on-out cross validation. At the moment the sampling is too sparse and everything is independent and not smooth at all. This is described in the intro of "Assessment of probability density estimation methods: Parzen window and Finite Gaussian Mixtures" http://www0.cs.ucl.ac.uk/staff/c.archambeau/publ/iscas_aa06.pdf http://www0.cs.ucl.ac.uk/staff/c.archambeau/publ/iscas_aa06.... (as well as many other places that just happened to be the first google result) email me if you want more details, I am ex-informatics.
- chriswait 12y ago
  This is massively useful, I was looking for a way to 'aggregate' results somewhat. I will read through this today and might send you a question or two later on if you don't mind.
  - tlarkworthy 12y ago
    Yeah send any queries over or I can go through it face to face if you want.
wambotron 12y ago
This is interesting, but I feel like I have no real knowledge of the situation to give a good vote. I voted 'yes' as I assumed, after looking at the map, that most Scots want it. It doesn't matter to me, though. I'm not Scottish!
- smcl 12y ago
  Interestingly "Scottish"ness doesn't necessarily matter for eligibility to vote in this referendum. I was born and raised there for 25 years but cannot vote as I'm not resident there. Those who live in Scotland can vote, regardless* of their nationality or how they identify. Which is of course correct and reflects what I'd like an independent Scotland to look like - immigrant friendly and diverse. * = within certain restrictions, something like a couple of years resident prior to the referendum. Don't quote me on that :)

moron4hire 12y ago

I wanted to see what you have so far, but since I'm not Scottish, I didn't want to vote first. I denied access to location data, received a message saying I couldn't vote without location, and then had no way to view the current data on the map.

chriswait 12y ago
Thanks for the feedback! You make a good point: We're asking before we're giving anything. Completely understandable to deny location services. Would your preference be to see the heatmap worldwide before being asked for your location?
- cwmma 12y ago
  or maybe just a shortcut to see results button
- moron4hire 12y ago
  Well, if you show the heatmap right away, it might skew your results. Perhaps if you had a third option "skip voting and view results".

nly 12y ago

Looks like a population map.

valarauca1 12y ago
Because it basically is. XKCD even did a comic about this http://imgs.xkcd.com/comics/heatmap.png http://imgs.xkcd.com/comics/heatmap.png
viraptor 12y ago
When the colours are summed up? I would hope it is - otherwise data is missing. But you can definitely see differences between cities when you zoom up. For example Glasgow seems to be a primarily "No" place, but Kirkcaldy is mostly for "Yes".

GordonS 12y ago

Doesn't work for devices that can't accurately provide location information. For example, I live near Aberdeen, but my ISP is in London - so this shows me as in London. Doesn't seem to be a way to override the 'detected' location?

chriswait 12y ago
You're right - we haven't done this yet as we anticipated most of the traffic to come from mobile devices. I will have a look an manually entering location today (now wondering if an address search box is better, or "drop a pin"?).
- jarofgreen 12y ago
  The location services on my phone are often off, or I may deny location permission until I see more what the site is. Bad assumption. (Good luck with project tho!)

maxbog 12y ago

I think you are publishing locations that are too specific. I can identify houses, from which the votes were cast. This information can be abused and is therefore dangerous. Please do not publish such information without informing the user loudly and clearly of the implications of voting.

chriswait 12y ago
I think you're absolutely right. I was considering adding some noise to the lat longs collected to mitigate this. We're looking at the copy (to better highlight what data is stored and displayed). Thank you for the feedback.
- Nowaker 12y ago
  Maybe place all the marks in the town center?
  - billybofh 12y ago
    That would diminish the usefulness of the map quite a bit. People will be interested in this on a ward-by-ward basis - I'm pretty sure. A small random 'shuggle' should be enough to help.

mrloop 12y ago

The colour blue for one of the result sets is a poor choice as the map is drawn in blue / turquoise. You need 2 colours for the results which are of a similar distinguishability against the map colour.

chriswait 12y ago
Good call, having a think about choosing other colours now. Thanks for taking a look!

chestnut-tree 12y ago

This is a nice idea. My UX suggestions: - display the heatmap first - have a prominent link saying something like: add your vote to the map. When someone clicks that link, ask them to enter their location (city or town and country), then ask them the same question the Scots will vote for in the referendum: Should Scotland be an independent country? Yes/No. I think the auto-detect location causes too many problems and may have privacy implications if, as another poster says, you can identify quite precise locations. What would be really nice would be a map similar to the Book Depository ("Watch people shop" map): http://www.bookdepository.com/live http://www.bookdepository.com/live In this case, you would display a pop-up each time someone voted with the name of their town or city and whether they'd voted Yes or No.

chriswait 12y ago
Hi, thanks for all the feedback. I love the bookdepository.com user experience with the live updates. We'll be working on this as we get closer to the date! Auto detection of location is proving to be a bit tricky - we're seeing a lot of votes clustered around internet exchanges in places.
mbesto 12y ago
> - display the heatmap first - have a prominent link saying something like: add your vote to the map. When someone clicks that link, ask them to enter their location (city or town and country), then ask them the same question the Scots will vote for in the referendum: Should Scotland be an independent country? Yes/No. One nitpick - don't show the heatmap first if you want correct poll results. This heavily biases the undecided people into "oh, everyone in Aberdeen says No...I guess I say No then too!"

smcl 12y ago

I'm afraid I can't see a thing, is it down due to the load coming from HN? Just a white screen with a sort of google-maps-ish compass in a box. I'm very keen to see something like this, with the referendum so close! edit: back up now

chriswait 12y ago
Really appreciate you taking the look. The site is still very much up, could you try reloading the page?

pmtarantino 12y ago

You should post it on /r/Scotland on Reddit!

chriswait 12y ago
I was planning to give it a day (give the server a rest), but somebody beat me to it: http://www.reddit.com/r/Scotland/comments/2ej5kk/scottish_independence_referendum_heatmap/ http://www.reddit.com/r/Scotland/comments/2ej5kk/scottish_in...

midhir 12y ago

Looks interesting, congrats on getting it shipped! A couple of things: 1) At any kind of scale, with smaller data-sets, the blue and red are overlapping in places to form purple and the user has to work out from the hue which is doing better. E.g. Edinburgh looks like it has a high blue content but I need to zoom in to see that really it's dominated by a massive blue blob on Grassmarket. 2) (Wonkish) The YES and NO buttons are stacked vertically with no margin. If the order isn't randomised already I'd consider doing that and separating the buttons. 3) I'm not from Scotland, but I did vote. If you're collecting that data also, why not show it? It'd be interesting for sure. We can often see Scotland across the water from our town, there are plenty of people here who consider themselves Scottish and others besides who have a dog in the fight. Looking forward to the 18th anyways. Yes or No, it'll be a hell of a party!

chriswait 12y ago
Hi there, thanks for the feedback! 1. I'm looking at the way google maps heatmaps lump results together at different zoom levels. If anyone has any experience with this, I'd love to hear about it. 2. This is exactly right. On the todo list now. 3. If you voted outside scotland (and the browser got a location), then it'll be displayed on the map in the place you voted from. Is this what you meant? Really appreciate all the comments.
- midhir 12y ago
  Ah cool! Starting to see some colours over here now. Belfast voting Yes and Dublin voting No.. Now there's a turnup for the books.

k-mcgrady 12y ago

You broke my browser (Safari). I said yes to giving location and it located me - then asked me again, and again and again. The only way out was to allow it to remember my decision for 24 hours. Then the site told me it couldn't locate me despite having a pin on my house.

chriswait 12y ago
Ah this sucks, sorry about that. I think I've found the source of this and it should be sorted now. Thanks for taking a look!
- k-mcgrady 12y ago
  Great thanks. I'll give it another try! Edit: Working now. I'd love to see a breakdown of votes (total yes/no and world yes/no vs scotland yes/no).
  - chriswait 12y ago
    Have since added global totals to the key at the top right, working on per-country totals over the next few days. Thanks again.

AndrewGreen 12y ago

To be clear, what is the question -- is it "Do you want independence", or is it "Do you want unity"?

Marazan 12y ago
Should Scotland be an independent country?
chriswait 12y ago
Hi Andrew. Thanks for the feedback, have updated the copy on the 'welcome' message to reflect this. (This message is only displayed to first-time users of the site.)
smackfu 12y ago
It seems like on the ground in Scotland, the sides are just "YES" and "NO" at this point.
smcl 12y ago
This is a good point but interestingly within Scotland (i.e. the target audience) the question doesn't need to be explicitly stated. People will have "Yes" (or "Aye") or "No" badges on their facebook profiles without any context. It's probably good that they added the question in though as I'm fascinated by the responses outside Scotland!

s_dev 12y ago

You should include the total respective votes with the key so the accuracy of the data can be compared to other polls. Maybe even a break down of the UK/ Ireland/ Europe/ Rest of World.

chriswait 12y ago
I love this idea. Will probably start with the global breakdown first. Thanks for taking a look for us!

pawn 12y ago

I don't know anything about Scottish politics, but as a generality, it looks like most larger cities vote Yes while the smaller towns have people voting No. Can anyone with insight explain why that'd be?

jarofgreen 12y ago

What is "vote every 24 hours and show your support" about?

touristtam 12y ago

I don't think a heatmap is the best representation for this kind of data: 1. if this is meant to show both votes, then a pie chart for each position (town/council) would be more effective 2. the blur is not clearly visible at all when zooming out completely 3. taking into account 'votes' from outside the country concerned is a bit confusing as pointed out in another comment, the rules for the referendum states that you have to be in Scotland to cast you vote. However, I had a look at the http://bubb.al/ http://bubb.al/ website and I can see how this could be used in a contained area (like an attraction park) or more generally if you could tie this up with a social media to show the mood of a user population in a wide area. On a side note, I am surprised you are not located near the Quarter Mile. ;)

chriswait 12y ago
Thanks for the feedback! Looking at each point individually: 1. We went with a map over a series of pie charts as we'd like to create something a bit more engaging for the user. 2. Agreed: The blur is causing me several issues, and is something I'm absolutely still working on. 3. Obviously these "votes" won't be counted, we're just trying to get a sense of how people in different part of the world feel about it. Thanks for taking a look at our site! Attraction parks are a use-case we're absolutely something we're looking into. We're currently based in a tech incubator called CodeBase (http://www.thisiscodebase.com/ http://www.thisiscodebase.com/). Quarter Mile is a beautiful building, but you can't beat being surrounded by other startups!

touristtam 12y ago

Just a side note. The map is continuously trying to locate me in Birmingham which I guess is my ISP's end-point connection to the internet, whereas I am actually in Edinburgh. I am using a few different extension in chrome and in firefox that normally block advert/analytics request, but this time the extensions are disabled to check for interferences. So how to you get the location? Because it isn't working for me.