Google Maps polygons with holes

The last couple of weeks I have been experimenting with Google Maps trying to draw filled polygons that look alright. I’m using matplotlib for making the polygons and I’ve figured out that the output from contourf(…) is like a plotting routine where you first get a polygon that should be filled with the current level and the following ones are holes in it.

Before I just draw them all, coloring the holes with a lower color. This forced me to sort the polygons according to size which worked ok but didn’t look good.

In Google Maps v2 there was something called encoded polygons but it seems as if they were removed in v3 (never supported by Chrome anyway).

Anyway, the correct way in v3 is to do like this:

  var paths = [[
    new google.maps.LatLng(38.872886, -77.054720),
    new google.maps.LatLng(38.872602, -77.058046),
    new google.maps.LatLng(38.870080, -77.058604),
    new google.maps.LatLng(38.868894, -77.055664),
    new google.maps.LatLng(38.870598, -77.053346)
  ], [
    new google.maps.LatLng(38.871684, -77.056780),
    new google.maps.LatLng(38.871867, -77.055449),
    new google.maps.LatLng(38.870915, -77.054891),
    new google.maps.LatLng(38.870113, -77.055836),
    new google.maps.LatLng(38.870581, -77.057037)
  ]];

  function initialize() {
    var map = new google.maps.Map(document.getElementById("map"), {
      zoom: 16,
      center: new google.maps.LatLng(38.8714, -77.0556),
      mapTypeId: google.maps.MapTypeId.SATELLITE
    });

    var poly = new google.maps.Polygon({
      paths: paths,
      strokeWeight: 3,
      fillColor: '#55FF55',
      fillOpacity: 0.5
    });

    poly.setMap(map);
  }

My project is now online at http://halvklart.se/ and here’s a screenshot:

There are some troubles with the polygons when applying the b-splines but I will try to take care of them in the near future.

I have briefly investigated Thrift and ProtoBuf but I’m still not sure that I will gain that much by switching from JSON. Also found something called BSON which is binary JSON. I think the next step will be to add some more parameters, wind direction is probably the hardest since I will have to draw the arrows myself.

 

Improving Google Maps polygons with b-splines

Google Maps is great, you get an extremely nice background map for free. I know that there are alternatives (Bing, OpenLayers, etc) out there but since I’m running Google App Engine it seems easier to go Google all the way.

I’m plotting polygons and polylines (that’s what weather is about) and it works great but my input data is kind of sparse so the polygons look very rough.

To improve them I’m using b-splines. Found a very nice article here. I just changed the javascript so it works with lat/lon-arrays and the output is an array of google.maps.LatLng.

function bspline(lats, lons) {
    var i, t, ax, ay, bx, by, cx, cy, dx, dy, lat, lon, points;
    points = [];
    // For every point
    for (i = 2; i < lats.length - 2; i++) {
        for (t = 0; t < 1; t += 0.2) {
            ax = (-lats[i - 2] + 3 * lats[i - 1] - 3 * lats[i] + lats[i + 1]) / 6;
            ay = (-lons[i - 2] + 3 * lons[i - 1] - 3 * lons[i] + lons[i + 1]) / 6;
            bx = (lats[i - 2] - 2 * lats[i - 1] + lats[i]) / 2;
            by = (lons[i - 2] - 2 * lons[i - 1] + lons[i]) / 2;
            cx = (-lats[i - 2] + lats[i]) / 2;
            cy = (-lons[i - 2] + lons[i]) / 2;
            dx = (lats[i - 2] + 4 * lats[i - 1] + lats[i]) / 6;
            dy = (lons[i - 2] + 4 * lons[i - 1] + lons[i]) / 6;
            lat = ax * Math.pow(t + 0.1, 3) + bx * Math.pow(t + 0.1, 2) + cx * (t + 0.1) + dx;
            lon = ay * Math.pow(t + 0.1, 3) + by * Math.pow(t + 0.1, 2) + cy * (t + 0.1) + dy;
            points.push(new google.maps.LatLng(lat, lon));
        }
    }
    return points;
}

There are some more things that you have to do, the original arrays have to be extended by adding the first 2 elements at the back and the last 2 at the front. Or else the polygon will look chopped.

Also the first point may have to be added at the end to close a polyline, a polygon will close itself.

This is what the picture looks like without b-splines:

And with b-splines:

There are some problems with incomplete polygons (polylines that enter and exit the area) but by skipping the first and last point they look ok.

Next step is to use Thrift for communication instead of JSON. Don’t know if it will actually make any difference but I have promised myself to learn either Google Protocol Buffers or Thrift.

Google App Engine ReferenceProperty and HTML5 local storage

The best thing with my job is that I work with the same things that I can spend hours doing in my free time. Too bad you don’t have 40 hours a week free time.

It’s been a while but I have finally made som progress.

I had some troubles with BigTable (the database that you use in Google App Engine). I put pretty large arrays with weather data in db.BlobProperty but when I read this back from the database GAE ran out of memory, even if I didn’t touch the blob. After reading up on this I found out that I had to use db.ReferenceProperty.

As always the manual is not that clear so here is some example code:

class ForecastData(db.Model):
    values = db.BlobProperty()

class Forecast(db.Model):
  firstGridPoint = db.GeoPtProperty()
  lastGridPoint = db.GeoPtProperty()
  increment = db.FloatProperty()
  parameter = db.StringProperty()
  forecast_data = db.ReferenceProperty(ForecastData)
  reference_time = db.DateTimeProperty()
  forecast_time = db.DateTimeProperty()
  insert_time = db.DateTimeProperty(auto_now_add=True)

I put my blob in a separate model and referenced it with a db.ReferenceProperty(ModelName). Below is an example for putting data in the Data Store.

    # Create the data object
    forecast_data = ForecastData()
    forecast_data.values = values

    # Put in in the database
    forecast_data = forecast_data.put()

    # Create the forecast object
    forecast = Forecast()
    # Reference the data (forecast_data is a key)
    forecast.forecast_data = forecast_data

And getting the data is done like this:

query = db.GqlQuery("SELECT * from Forecast where forecast_time=:1", forecast_time)
forecast = query.fetch(1)
if forecast:
    forecast_data = Forecast.forecast_data.get_value_for_datastore(forecast[0])
    forecast = ForecastData.get(forecast_data).values

I get the forecast object from the database with a GQL query. The referenced property can be fetched with the get_value_for_datastore method.

After this the application is much faster.

To minimize the data transfered I’m using HTML5 local storage (a very good guide to html5 can be found here).

To put something in the local storage:

window.localStorage.setItem('key', value);

and to get it back (even if the browser have been closed):

window.localStorage.getItem('key');

This is a very simple key/value store. Other useful commands are clear() which clears all saved values.

I’m hoping to launch the site for others to try out very soon but I want to get some more features in place.

Until then here is an up-to-date screenshot:

Weather and clouds

Ever since I wrote my master thesis I have had an interest in weather. My thesis was about metadata updates in a real time database at the Swedish Meteorological and Hydrological institute. The thesis was mainly about weather stations that deliver observations used when running weather models.

After my thesis I ended up as a consultant but for the last 2,5 years I’m back at SMHI doing various work, mostly backend. I have worked with real time deliveries of weather along with processing, visualisation etc. I worked with Diana (diana.met.no), a weather workstation software developed by Met.no.

Weather is a good thing since it is to interest to almost everyone. Telling someone that you’re a system developer is not a good conversation starter but saying that you work with weather will at least lead to a couple complains about how bad the forecasts are. From a system engineering point of view weather is good since there are huge quantities of data to process and it is a good source for testing new technologies.

Last fall I did a couple of tests using Python + CUDA + GRIBAPI for reading grib data (gridded binary is the format used for weather) and using CUDA to run marching squares and render images. As always there are to little spare time to finish all the small projects but at least I got a feel for CUDA and I hope to be able to use it in the future.

On to the clouds. 2 years ago I went to EclipseCon and learned a lot about EC2, both the founder of Smugmug and Stackoverflow praised it so after coming home I started using it at work. Today we have a couple of machines running in EC2 together with S3 for storage and RDS for databases. My primary server at home has handed over a lot of the work to an free tier EC2 instance.

I started using Python last fall and I like it a lot, being a shell scripting ninja I like regular expressions and {ba,k,c}sh but reusing scripts is not that easy but Python is perfect for this.

Getting to the point, my last project is to learn Google App Engine and I will do this by building a weather viewer using free weather data from NOAA. Python and probably some shell scripting will download the data and upload it to GAE and then I will use Django to render a site with Google Maps and my weather on it.

I want to build a free service that display the raw weather and I will build some kind of API so that people can get forecast values for latitude and longitude points.

Why? Mostly to learn Google App Engine but also since I know that there are people in need of such an API.