Published on GEOG 585: Web Mapping (https://www.e-education.psu.edu/geog585)

Home > Lessons > Lesson 5: Building tiled maps with FOSS

Lesson 5: Building tiled maps with FOSS

The links below provide an outline of the material for this lesson. Be sure to carefully read through the entire lesson befor returning to Canvas to submit your assignments.
Note: You can print the entire lesson by clicking on the "Print" link above.

Overview

This week's lesson focuses on tiled web maps. Tiles are relatively small square-shaped "chunks" of data (either rasterized map images or raw vector coordinates) that have been pregenerated by the server and stored in a directory called a cache. When web users navigate the map, the server can just hand out the tiles rather than generating the map on the fly.

In this lesson, you'll learn the pros and cons of tiled maps, as well as strategies for building and maintaining a tile cache. You will learn about traditional rasterized tiled image formats, as well as a newer generation of tiles that store vector coordinates. Because rasterized image tiles have been around much longer and have mature support from tile generating engines, tile servers, and clients, most of the lesson content focuses on these types of tiles. The newer vector tile format is discussed near the end of the lesson content.

There are two walkthroughs associated with this lesson, both of which involve rasterized image tiles because these currently have the most mature tools for end-to-end FOSS workflows. The first walkthrough shows how to create a simple cache of your Philadelphia neighborhoods map using the GeoWebCache software that is integrated into GeoServer. In the second walkthrough, you'll use TileMill and a markup language called CartoCSS to create a Philadelphia basemap cache with some nicer cartography than you would get using GeoServer.

Objectives

  • Describe the advantages of tiled web maps and identify when it is appropriate to use them.
  • Recognize strategies and techniques for creating and updating large tiled web maps.
  • Describe the differences between rasterized image tiles and vector tiles, and the reasons that each might be used
  • Create tiles for a WMS using GeoWebCache.
  • Use best practices in multiscale map design to create a tiled basemap using TileMill.
  • Discuss hosting options for tiled maps. Unpack and upload your tiled map to your own web space (on PASS).

Checklist

  • Read the Lesson 5 materials on this page.
  • Complete the two walkthroughs.
  • Complete the Lesson 5 assignment.

Why tiled maps?

As mentioned in previous lessons, the earliest web maps were typically drawn on the fly by the server, no matter how many layers were available or requested. These are the types of maps you just created using GeoServer and WMS. As you may have noticed, the symbol sets and labeling choices for this type of map are relatively limited and complex to work with. In fact, for many years, web cartographers had to build a map with minimal layer set and simple symbols to avoid hampering performance. In many cases, a cartographer was not even involved; instead, the web map was made by a server administrator tweaking XML files that defined the layer order, symbol sizes, and so forth. This was the case with both open specification web services (like WMS) and proprietary web services (like Esri ArcIMS).

Part of this approach stemmed from early efforts to make web GIS applications look exactly like their desktop counterparts. Sometimes these are referred to as “Swiss Army Knife applications” because they try to do everything (you may know one!). People expected that in a web GIS they should be able to toggle layer visibility, reorder layers, change layer symbols on the fly, and do everything else that they were accustomed to doing on the desktop. Ironically, this mindset prevailed at a time when web technology was least suited to accommodate it.

In the mid-2000s, after Google Maps, Microsoft Virtual Earth (now Bing Maps), and other popular mapping applications hit the web, people started to realize that maybe they didn't need the ability to tinker with the properties of every single layer. These providers had started fusing their vector layers together in a single rasterized image that was divided into 256 x 256 pixel images, or tiles. These tiles were pregenerated and stored on disk for rapid distribution to clients. This was done out of necessity to support hundreds or thousands of simultaneous users, a burden too great for drawing the maps on the fly.

The figure below shows how a tiled map consists of a "pyramid" of images covering the extent of the map across various scales. Tiled maps typically come with a level, row, and column numbering scheme that can be shared across caches to make sure that tile boundaries match up if you are overlaying two tile sets.

 Tile Pyramid
Figure 5.1 Tiled web maps take the form of a pyramid where the map is drawn at a progressive series of scale levels, with the smallest (zoomed out) scales using fewer tiles.

Cartographers loved the tiled maps, because now they could invest all the tools of their trade into making an aesthetically pleasing web map without worrying about performance. Once you had created the tiles, you just had a set of images sitting on disk, and the server could retrieve a beautiful image just as fast as it could retrieve an ugly one. And because the tiled map images could be distributed so quickly by a web server, Google and others were able to employ asynchronous JavaScript and XML (AJAX) programming techniques to retrieve the tiles with no page blink as people panned.

This was revolutionary. Which would you rather have: a slippy map with stunning cartography and no layer control, or a clunky and ugly map with the ability to reorder layers and adjust the color of a school? Some longtime GIS geeks had to stop to think about this, but for the common web user, the choice was a no-brainer.

Within a year or two of Google Maps' release, commercial GIS software began offering the ability to build map tiles. For many, ArcGIS Server was desirable because the map could be authored using the mature map authoring tools in ArcMap; however, cost was a concern for some. Arc2Earth was another commercial alternative. The free and open source Mapnik library could also build tiles, but it wasn't until recent years that projects like TileMill wrapped a user-friendly GUI around Mapnik.

 Tiles from OpenStreetMap data, rendered by MapQuest
Figure 5.2
Credit: Tiles from OpenStreetMap data, rendered by MapQuest

Tiled maps were the only model that could reasonably work for serving complex web maps to thousands of simultaneous users. However, they eliminated the ability for users to change layer order or symbols. People started working around this by serving out their general-purpose basemap layers as tiles and then overlaying a separate layer with thematic information. The general-purpose basemap tiles could be re-used in many applications. The thematic layers could also be tiled if the data didn't change too quickly or cover too broad an area at large scales. For example, if you examine Google Maps with a developer tool, you will see that the basemap and the thematic layers (such as Panoramio photographs) are both retrieved as tiles.

 Photos appearing as tiles in Google Maps
Figure 5.3 The thematic layer of Panoramio photos is brought into Google Maps as predrawn tiles. This is evident when viewing the layer in Firebug.

 

Making the decision to build and maintain tiles

If you want faster navigation of your basemap or you feel that more than a couple of users will be requesting maps simultaneously from your server, you should create a tile cache of your basemap. You may also choose to cache thematic layers if their features are not constantly changing attributes or position.

In both cases, be aware that the tile cache represents a snapshot of your data at the time the cache was created. To put it more bluntly, your tiles are “dumb images” that will not automatically update themselves when the back end data changes. You are responsible for periodically creating new tiles in order to update the map. With large caches, sometimes server administrators target the cache updates at just the changed areas rather than rebuilding tiles for the entire map. This requires keeping some kind of log about which places were edited, or comparing “before” and “after” versions of a dataset.

Is there an existing tile cache that meets your needs?

Building a tiled basemap requires lots of rich data, high-end map authoring software, cartographic skills, and potentially enormous amounts of time and disk space. You may still need to do it at one point or another, which is why you will get a taste of this experience in Geog 585. However, because of these challenges, general-purpose web mashups often use tiles made by somebody else. OpenStreetMap is an attractive option if you want free tiles with no restrictions. If you want to use Google's, Microsoft's, or Esri's tiles, you may be able to use them for free or you may have to pay, depending on the nature of your map (commercial or not for profit) and how many people use your app. Other companies such as CloudMade and MapBox have marketed their own versions of tiles using OpenStreetMap data.

If you are going to build your own basemap, it's helpful to have an experienced cartographer on staff who is experienced with designing at multiple scales. The symbols, colors, and details must be adjusted appropriately at each of the scales where tiles are created. Tiled basemaps can quickly get complicated with layer and labeling scale suppression rules. Cartographers may also need to design one set of tiles to stand on its own and another set of tiles to overlay remotely sensed imagery, a task that requires very different colors and symbols.

Projection

If you are going to overlay your tiles with any of the tiles from OpenStreetMap, Google, Microsoft, or Esri (or even just attempt to look like them), you must warp your most precious GIS data into a modified spherical Mercator projection that was created solely for the convenience of fitting the world onto a set of square tiles. GIS purists balked at this idea and predicted it would fail to achieve mass uptake, but many people (at least at mid latitudes) now hold their nose and move forward with it.

Be aware that Esri, Google, and other organizations have used other code numbers and variations of this projection in the past: things can get very confusing if you are using older software or APIs. In the past couple of years, people seem to have standardized on EPSG:3857 although even the subparameters of this projection can be interpreted in diverse ways that lead to offsets. As a side note, can you figure out the humorous reason that Google once used the code 900913 for this projection?

Even when you display your maps in EPSG:3857, you do not ever want to perform measurements in this projection. The results will be largely skewed even at mid latitudes. It's best to make sure that geometries are projected into a more local coordinate system before performing any measurements. The Esri blog post Measuring distances and areas when your map uses the Mercator projection [1] gives a good overview of the problem and an example solution that uses the ArcGIS API for JavaScript, although the concepts can also be applied with FOSS APIs.

Scales

Not only must you match the projection in order to overlay, you must also match the scales. These are not nice well-rounded numbers; rather, they were derived mathematically from the starting point of putting the whole world on a 2x2 grid of tiles. For example, one of the scales is 1:36,111.98 and as you zoom in, the next one is 1:18,055.99. So much for your simple USGS 1:24000 series! Partly for this reason (and partly because many laypeople don't understand map scales), the common web map scaleset is often referred to with simple numbers such as “Level 14,” “Level 15,” etc., that increase as you zoom in. You just have to get a feel for which levels correspond to national scale, provincial scale, city scale, neighborhood scale, etc. The table in the Bing Maps Tile System [2] article is helpful for this.

Strategies for creating and serving map tiles

Map tiles often have a simple folder structure, which makes them easy to serve. However, they can become complex to manage due to their sheer size and number. There are several different ways people have devised to serve map tiles:

  • Put them in a folder structure on your web server, and let people pull them directly. In this approach, you just organize the tiles as individual image files under a folder structure representing the scale level, row, and column. Many mapping APIs can use tiles when presented with a URL structure with variables representing the consistent position of the level, row, and column. For example, when using a tiled layer in the Leaflet API, you must provide a URL in the form http://{s}.somedomain.com/blabla/{z}/{x}/{y}.png, where z is the zoom level and x and y are the column and row.
  • Make the tiles available through a web service. In this approach, the web service exposes parameters for you to request a tile based on a particular level, row, and column; however, the back-end infrastructure is a little more hidden. This approach has slightly more overhead (and therefore time lag) than exposing the tiles directly through a folder. A variation of the WMS specification called WMTS is designed for serving tiles and works in this manner. You can also see this pattern at work when you navigate around Google Maps using Firebug or some other developer tool. The following tile URL for Google Maps reveals nothing about the back-end folder structure for the tiles, although you can see the level, row, and column variables at work if you look hard enough: https://mts0.google.com/vt/lyrs=m@241289412&hl=en&src=app&x=74&y=96&z=8&s=Galile [3]

Strategies for creating tiles

At large (zoomed in) scales, the number of tiles to cache can be overwhelming, especially if you are covering a broad area such as a state or country. The irony is that at such large scales, many tiles will convey very little information. Zooming in to a neighborhood block scale at 1:2250 may show plenty of interesting features, but pan out into the desert or ocean at the same scale, and the tile may be completely blank. Do you want to spend the hours creating these tiles and gigabytes of disk space to store them?

In these situations, you may want to find software that can create tiles on demand, meaning at the time they are first visited by a user. The first person to navigate to a region will need to wait for the server to create the tiles, but subsequent visitors will enjoy the full benefits of created tiles. The most popular areas will fill up with tiles, but you won't spend resources creating and storing tiles that are never visited. Obviously, this approach varies in effectiveness based on how fast the server can actually draw the tiles on demand.

An alternative is to use a “Data not available” tile to denote areas where tiles have not been created. Web map administrators are sometimes loathe to do this, but it is often a common enough practice that lay users blame themselves when they see the tile (“Oops, I zoomed in too far!”) rather than the administrator (“Why didn't they make the map available at this scale?”).

The best approach may be to strategically create a subset of the most interesting tiles and leave the less interesting tiles to be created on demand (or returned using a “Data not available” tile). As a geographer, it may hurt your soul to call any place “less interesting,” but the bitter truth is that not all tiles of the map experience an equal number of views. Fisher (2007) showed how early visitors to Microsoft Virtual Earth (now Bing Maps) stuck mostly to the big cities, coastlines, and transportation corridors. Quinn and Gahegan (2010) built models taking into account these patterns, showing how the majority of map requests could likely be satisfied by creating a fraction of the full number of tiles that it would ordinarily take to cover the full rectangular extent of the map. Their models were cobbled together using datasets like buffered roads, coastlines, and points of interests, but more recent feeds from social media such as geotagged tweets and Flickr photos may prove even more accurate in revealing the most interesting regions of the map for the majority of users. Note that some types of specialized maps (for mineral exploration, or wilderness conservation) may have very different usage patterns than general-use basemaps.

The ability to selectively cache a subset of tiles like this depends on the tile creation software's flexibility in allowing the administrator to designate custom regions for caching. Most software just allows the submission of a rectangular bounding box for tile creation; however, interesting areas of the map such as cities and coastlines are usually not shaped like rectangles. If you identify an irregularly-shaped region where you want to create tiles, you may have to abstract it into a series of rectangles and run multiple tile creation jobs, using each rectangle as an input. If you are luckier, your tile creation software will accept a spatial dataset (such as a shapefile) as a bounding region.

Creating tiles with FOSS

Creating tiled web maps is a common task that has been addressed by various FOSS packages. The most accessible one for you at this point is GeoWebCache, because it's integrated directly into GeoServer. Others include TileCache and TileStache.

The Mapnik library is a FOSS tile creation library that binds to Python and other languages. It allows a lot of advanced drawing options not found in your typical WMS layer. Working with Mapnik typically required some Linux knowledge and some trial and error; however, a couple of years ago, the for-profit company Mapbox released an open source program called TileMill that can run on Mac and Windows and puts a nice GUI around Mapnik. This simplifies the cartographic process and puts Mapnik within the scope of Geog 585. In the second walkthrough in this lesson, you'll use TileMill to create a basemap of Philadelphia using some of the layers you processed earlier. Of course, feel free to also explore Mapbox Studio if you are interested, and share your experience with it on the forums.

References

  • Fisher, D. (2007). Hotmap: Looking at geographic attention. Visualization and Computer Graphics, IEEE Transactions on, 13(6), 1184-1191.
  • Quinn, S., & Gahegan, M. (2010). A predictive model for frequently viewed tiles in a web map. Transactions in GIS, 14(2), 193-216.

Vector tiles: the next generation of tiled maps?

Although the rasterized tile sets we have discussed in this lesson are able to deliver nice-looking maps in a relatively rapid format, they can be cumbersome to keep updated and they require enormous amounts of computing resources at large map scales. To work around these challenges, a data storage format called "vector tiles" has gained popularity in the past several years. Mapbox has led development efforts on vector tiles and has shared a vector tiles specification [4] under a Creative Commons license. 

Vector tiles are exactly what you would guess: they store chunks of vector data instead of storing a map image. The idea behind vector tiles is that it is more efficient to keep data styling separate from the data coordinates and attributes. The client can use a predefined set of styling rules to draw tiles of raw vector coordinate and attribute data sent by the server. This allows the restyling of data on the fly, which is another serious limitation of rasterized tiles. Think about it: If you want to change the shade of green used to draw parks with your rasterized tiles, you must rebuild every tile containing a park. If you want to do the same thing with vector tiles, you just update your styling instructions in one place and the tiles themselves stay the same. Other display operations such as rotating the map also become easier to implement with vector tiles.

Vector tiles are designed to be small on disk, and employ a number of optimization approaches designed to reduce the amount of characters needed to store the geographic data and attributes, some of which are described in this video by Mapbox engineer Dane Springmeyer [5]. He also introduces a product called Mapbox Studio [6] which works with vector tiles only and is being promoted by Mapbox as a replacement for Tile Mill. The .mbtiles file format, which originally stored rasterized tiles, now only stores vector tiles when exported from Mapbox Studio.

In reality, there continue to be use cases [7] for vector and rasterized tile formats, although it is likely that a number of organizations will see performance benefits from rebuilding some of their originally rasterized tile sets as vector tiles in the future. This is even more likely as popular commercial software packages such as ArcGIS introduce tools to work with the Mapbox vector tile specification, a strategic decision that Esri announced in a 2015 blog post [8]. Open source tools are also recognizing the staying power of vector tiles, exemplified by the VectorTile layer format [9] built into OpenLayers 3 and plugin support for Mapbox vector tiles in Leaflet [10]. At the time of this writing, QGIS does not natively support viewing vector tiles, although some have been developing plugins [11] for this purpose.

Walkthrough: Creating tiles with GeoServer using GeoWebCache

Suppose you're satisfied with the layers and symbols in your WMS, but you want it to draw faster and be available to many simultaneous users. In this situation, it might make sense to use GeoWebCache to create your tiles, because GeoWebCache is built directly into GeoServer. In this walkthrough, you'll use GeoWeb Cache to create a tile cache for the Philadelphia NeighborhoodMap group layer WMS that you published in the previous lesson.

  1. Start GeoServer and open the GeoServer Web Admin page.
  2. Use the Layer Preview link to preview your geog585:NeighborhoodMap group layer (the one with the green icon). Use the OpenLayers preview so that you can zoom and pan around. Take note of the performance and appearance of the map. You'll see the labels repositioned on each pan, a sure indication that tiles are not being used yet.
  3. In the GeoServer Web Admin page, click the Tile Layers link.
  4. Click the geog585:NeighborhoodMap link, and then select the Tile Caching tab. This is where you can set up parameters for caching of your layer. If you stopped here, your cache would be created on demand. In our case, we actually want to pregenerate tiles for most of the levels so that they won't need to be built on demand. GeoWebCache calls this "seeding" the cache.
  5. Click Tile Layers again and in the row for geog585:NeighborhoodMap, click Seed/Truncate.
     Seed or truncate cache link
    Figure 5.4
  6. Scroll down, and fill out the form to create a new task as shown in the image below. Notice that your tile creation task will create PNG images using the Google Maps tiling scheme (900913).
     Create new seed task
    Figure 5.5
  7. At the bottom of the form, click Submit. Although you can't see it, your computer is busy drawing your map over and over at different scales. This can tax the computer's memory and CPU resources, and you may see a performance impact on other applications while the caching is occurring. If you want to see your processor at work, open Windows Task Manager and look at the CPU and memory resources being used by java.exe.
  8. Every 30 seconds or so, click the Refresh List link to see the progress of your cache. When you click this link and the task status disappears, it means your cache is complete.
     Refresh list
    Figure 5.6
  9. In the GeoServer Web Admin page, click Tile Layers and use the dropdown list to preview your cache in EPSG:900913 using the PNG format (see image below). This time when you pan around, you should see the map appearing instantly. You can verify that the tile cache is being used if the labels do not change position as you pan.
     Preview tiled layer
    Figure 5.7

    Caution: Make sure you are using the Tile Layers preview and not the Layer Preview preview. The Tile Layers preview uses a slightly different URL for the layer that indicates the tile cache should be used. Also, do not worry if the cache is reported on the Tile Layers page as N/A or 0.0 B in size. This seems to be normal, even though you have now built the cache. 

Although performance is improved with the tile cache, you may notice some duplicate labels appearing. This is a difficult problem to avoid with map tiling, because each tile does not "know" about the labels on the adjacent tiles. To mitigate this problem, tile caching software typically draws an area much larger than a tile and then cuts it up into individual tiles. GeoWebCache calls this large area a "metatile" (Esri calls it a "supertile"). If you like, you can experiment with adjusting the metatile size; although duplicate labels can still appear at the metatile boundaries, the duplicates will be fewer and farther between. You may also find that the settings and options in the next walkthrough with TileMill make it easier to get the labeling you want.

Walkthrough: Creating tiles with Mapnik using TileMill

In this walkthrough, you will use Mapnik (wrapped by TileMill) to create a general-use basemap of Philadelphia. The idea is that you will be able to overlay thematic layers on top of this map in lessons to come. The data for this walkthrough is the base data for Philadelphia that you preprocessed in Lesson 3. If you followed all instructions, you should have this data in a folder called c:\data\Philadelphia or something similar. It should be using the mercator projection EPSG:3857.

Mapnik [12] is a FOSS map-drawing engine that is often used for the purpose of making sets of map image tiles. Mapnik incorporates techniques like antialiasing that make the edges of lines and labels appear smooth, not pixelated. It is not tied to the overhead of any other GIS software framework, a fact that improves its performance. Various types of file-based data and spatial databases are supported as data sources for the maps.

To use Mapnik, you define a set of data sources and then connect them to a bunch of styling rules. You can set up the styling rules using an XML file [13], or you can write them programmatically [14] using a language like Python. Once you've configured the data sources and style rules, you can instruct Mapnik to export pictures of the map.

This is easy in theory but is not the simplest process for a beginner, especially if your programming skills are thin. To work with Mapnik in this course, you'll use a program called TileMill that makes the process a lot easier. TileMill puts a GUI around Mapnik, allowing you to browse to the datasets you want to use and see the results of your work as you apply styling rules for each layer. Instead of using an XML file or code to define the rules, you'll use a fairly intuitive language called CartoCSS that borrows from the structure of web stylesheets.

TileMill is FOSS, but was developed by Mapbox to integrate with the company's for-profit tile hosting service. This is evident when you export tiles with TileMill and the resulting file has the extension .mbtiles. If you don't want to host the tiles on Mapbox's servers, you can "unpack" the tiles out of the .mbtiles file and host them as individual images on your own web server. In this walkthrough, you'll unpack the images and host them on your Penn State PASS web space. Then you'll test the tiles by supplying their URL structure to a web map.

As you complete this walkthrough, you may see occasional software messages that Tile Mill is not under active development and that Mapbox has shifted its focus to Mapbox Studio. Tile Mill continues to be available as a FOSS tool for building tile sets of rasterized map images. In contrast, Mapbox Studio is geared towards creating vector tiles which Mapbox will then host in exchange for a fee. Be aware that Mapbox Studio-generated .mbtiles files you encounter professionally may contain vector tiles instead of containing rasterized images like the Tile Mill-generated .mbtiles files we will use in this walkthrough.

Installing TileMill and designing the map

  1. Download and install TileMill from the following page: https://www.mapbox.com/tilemill/ [15]. Take all the defaults. When you install TileMill, Mapnik is installed for you as well.
  2. Launch TileMill from your Windows Start menu by clicking TileMill > Start TileMill.
  3. Click New Project and fill out the information as in the image below. Then click Add.
     New project in TileMill
    Figure 5.8
    If you were making a cache of raster imagery, you might want to use the lossy jpeg image format to keep the cache size down. Since we are making a vector map, the png format is fine.

    Note here that having a map with vector data does not mean you will need to use vector tiles. The vectors in our map will be rasterized by the tile creation process, meaning that users will just be seeing a bunch of "pictures of the data".
  4. In your project list, click PhillyBasemap and take a look at your canvas. In TileMill, your map preview appears on the left and your CartoCSS code appears on the right. The map updates whenever you save your CartoCSS code.

    Let's see how this process works by changing the background color from blue to white.
  5. Set the background-color property to #FFFFFF (the HTML hex code for white) as shown below, and click Save.
    Map {
       background-color: #FFFFFF;
    }
    The background should update to white as soon as you save your code. Now, let's add some layers and follow this process to style them.
  6. Click the layers button, and click Add Layer.
     TileMill Add layer button
    Figure 5.9
  7. Add the city_limits.shp shapefile as a layer using the exact properties as shown below.
     Add city limits layer in TileMill
    Figure 5.10
    A couple of things here deserve mention. First, TileMill allows you to set varying levels of granularity on your styling rules, using the concept of classes and IDs. For example, if you wanted to set some general rules for all streets, you could set up a class for all streets, and if you wanted to set some more specific rules for a certain subgroup of streets, you could use an ID for that subgroup. You're only going to set one style on the city limits layer, so these instructions tell you to set the Class and ID to the same name. In fact, each layer in this tutorial will use an identical ID and Class name for simplicity.

    Also, projection code 900913 must be defined as the spatial reference when you are using files that you originally projected into EPSG:3857 using OGR. The two projections are equivalent, but if you leave TileMill to autodetect the spatial reference, you will see an offset from Google's tiles due to the way the projection parameters are interpreted.
  8. The starting view at level 2 is zoomed too far out for work with Philadelphia, so use the Zoom to extent button to get your map zoomed in to the city limits layer.
     Zoom to layer
    Figure 5.11
  9. The default #CityLimit styling makes the city look like a big park, so change the style to a simple boundary using the following code:
    #CityLimit {
      line-color:#88789e;
      line-width:3;
    }
    When you save, you should see the following:
     City limit styled in TileMill
    Figure 5.12
    Now that you've got the process down for adding and styling a layer, the instructions are going to start moving faster. I will provide information about which layers to add, and some basic code to style them. Make sure the ID you add for your layer matches the ID that I use in the code (remember the ID is the name following the # sign, such as #CityLimit or #Waterways).

    If you don't like my styling, you can try tweaking it to your liking, but be careful to maintain legal syntax. The Carto button provides reference documentation about how to use the different properties.

    Don't spend too much time making major changes to this map's styling, because you'll get a chance to make your own map during this week's assignment. After you finish the walkthrough, you can use this Philadelphia map for further practice and experimentation.
     TileMill Carto button
    Figure 5.13
  10. Add waterways.shp as a layer, and add the following code to style it. The position where you paste this block of code into your CSS doesn't matter.
    #Waterways {
      line-width:1;
      line-color:#89aceb;
    }
  11. Add natural.shp as a layer, and add the following code to style it.
    #NaturalFeatures{
      [type='park']{
      polygon-opacity:1;
      polygon-fill:#ae8;
      }
      [type='riverbank']{
      polygon-opacity:1;
      polygon-fill:#89aceb;
      }
      [type='water']{
      polygon-opacity:1;
      polygon-fill:#89aceb;
      }
    }
    Notice above that you can add query terms if you only want to display a subset of features in the dataset. We are displaying just the parks, riverbanks, and water features, even though there are a lot more types of features in this shapefile.
  12. Add Neighborhoods.shp as a layer, and add the following code to style it.
    #Neighborhoods[zoom>12] {
      text-name:[NAME];
      text-face-name:"Arial Black";
      text-fill:#88789e;
      text-size: 12;
      text-character-spacing: 2;
      text-transform: uppercase;
    }
    In this case, we only specify text properties. This has the effect of hiding the neighborhood boundaries so that just the label of the neighborhood name is shown. This is more fitting to neighborhoods anyway, since they don't tend to have rigid boundaries.

    When you label a layer in TileMill, you need to specify the name of the field in your dataset that contains the label text. In this case, you want to label based on the NAME field, therefore you put text-name:[NAME].

    Also, notice that you can set a condition on the layer so that it only appears in when you zoom in beyond a certain level. Zoom in past level 12 to make sure this is working correctly and that you see the neighborhood labels.
     Neighborhoods viewed in the TileMill map
    Figure 5.14
  13. Add roads.shp as a layer, and give it the ID and class name of MajorRoads. Then add the following code to style it.
    #MajorRoads{
      [type='motorway']{
       line-width:3;
       line-color:#606060;
      }
      [type='trunk']{
       line-width:3;
       line-color:#606060;
      }
      [type='primary'] {
       line-width:2;
       line-color:#838383;
      }
    }
    This will symbolize just the major roads. OpenStreetMap has a lot of different road types, and there are different ways you could apply styling here. In our situation, we will display minor streets by adding all the roads again and putting them beneath the major roads. This introduces some redundancy but keeps the code simpler.
  14. Add roads.shp as a layer again, but, this time, give it the ID and class name of Roads. Then, add the following code to style it.
    #Roads[zoom>12]{
      line-width:1;
      line-color:#b6b6b6;
    }
    		
    #Roads[zoom>14]{
      line-width:1;
      line-color:#b6b6b6;
      text-name:[name];
      text-face-name:"Arial Regular";
      text-fill:#838383;
      text-size: 11;
      text-placement: line;
      text-min-path-length:100;
      text-avoid-edges:true;
      text-min-distance:50;
      text-dy: 6;
      text-max-char-angle-delta: 15;
    }
    This puts some labels on the roads when zoomed in.
  15. Add railways.shp as a layer, and add the following code to style it.
    #Railroads{
      line-width:1;
      line-color:#d2bcb0;
    }
    		
    #Railroads[zoom>15] {
      ::line, ::hatch { line-color: #d2bcb0; }
      ::line { line-width:1; }
      ::hatch {
        line-width: 4;
        line-dasharray: 1, 24;
      }
    }
  16. Put your layers in the following order (by clicking and dragging to the left of the layer name), and save the map.
     TileMill layer order
    Figure 5.15
    Zoomed out, your map should look like this:
     TileMill map at zoom level 11
    Figure 5.16
    Zoomed in, your map should look like this:
     TileMill map viewed at zoom level 15
    Figure 5.17
    Your final code should look something like this, although the layers may be listed in a different order.
    Map {
      background-color: #FFFFFF;
    }
    
    #CityLimit {
      line-color:#88789e;
      line-width:3;
    }
    
    #Waterways {
      line-width:1;
      line-color:#89aceb;
    }
    
    #NaturalFeatures{
      [type='park']{
      polygon-opacity:1;
      polygon-fill:#ae8;
      }
      [type='riverbank']{
      polygon-opacity:1;
      polygon-fill:#89aceb;
      }
      [type='water']{
      polygon-opacity:1;
      polygon-fill:#89aceb;
      }
    }
    		
    #Neighborhoods[zoom>12] {
      text-name:[NAME];
      text-face-name:"Arial Black";
      text-fill:#88789e;
      text-size: 12;
      text-character-spacing: 2;
      text-transform: uppercase;
    }
    		
    #MajorRoads{
      [type='motorway']{
      line-width:3;
      line-color:#606060;
     }
      [type='trunk']{
      line-width:3;
      line-color:#606060;
     }
      [type='primary'] {
      line-width:2;
      line-color:#838383;
     }
    }
    		
    #Roads[zoom>12]{
      line-width:1;
      line-color:#b6b6b6;
    }
    		
    #Roads[zoom>14]{
      line-width:1;
      line-color:#b6b6b6;
      text-name:[name];
      text-face-name:"Arial Regular";
      text-fill:#838383;
      text-size: 11;
      text-placement: line;
      text-min-path-length:100;
      text-avoid-edges:true;
      text-min-distance:50;
      text-dy: 6
      text-max-char-angle-delta: 15;
    }
    		
    #Railroads{
      line-width:1;
      line-color:#d2bcb0;
    }
    		
    #Railroads[zoom>15] {
      ::line, ::hatch { line-color: #d2bcb0; }
      ::line { line-width:1; }
      ::hatch {
       line-width: 4;
       line-dasharray: 1, 24;
     }
    }

Exporting and extracting the tiles

Once you've finished the map design phase and your map looks good at each scale, you can start thinking about generating the tiles. For large maps, plan for this to take some time, taking into account some of the factors discussed earlier in the lesson such as the shape of the map and the scale levels you choose to generate.

This section of the walkthrough explains how to generate tiles of your Philadelphia basemap, unpack them, and host them on your web space. You will then test them out in a web page.

  1. Make one minor change in your code to increase the default buffer size around the metatile [16]. This helps avoid your very wide labels from being cut off when the tiles are drawn. You add this near the top of your code, in the Map section, like this:
    Map {
      background-color: #FFFFFF;
      buffer-size: 512;
    }
  2. In TileMill, click the Export button and choose MBTiles. This is MapBox's format for storing all the tiles in a SQLite database. We'll eventually unpack them as individual PNG images.
  3. Define the area and origin point for the cache. To do this, use the left-hand map view to zoom in to Philadelphia. Then hold down the Shift key and drag the left mouse button to draw a box around the Philadelphia city boundary. Make sure you do this at a fairly large scale (zoomed in) so that you don't create tiles for a lot of peripheral white space.

    When you have zoomed the map to the area that you want, right-click in the middle of your map to place the center point for the tiles.

    Your map should look like this:
     TileMill area of interest
    Figure 5.18
  4. In the right-hand menu, set your zoom slider to cover levels 0 through 17 as shown below. Notice how the number of potential tiles is affected by moving the slider, especially at the larger scale levels.
     Tile Mill zoom slider
    Figure 5.19
    Don't change any of the other default settings on this panel.
  5. Click Export, and note the progress bar giving you an idea of how much time is remaining to finish building your tiles. You can move on and do a few of the next steps while the tiles are being created, although, if you open Windows Task Manager, you'll see that a process called node.exe is using up most of your CPU power while it draws the tiles.

    To unpack these tiles, we'll be using a utility called MBUtil. You need to go download this from its home on GitHub (an online repository where developers often like to share their code with the world).
  6. Go to the MBUtil page on GitHub [17] and click Clone or download > Download ZIP. Save the zip file to your computer.
  7. Extract the contents of the zip file into a very easy path to remember, such as c:\mbutil.
  8. MBUtil requires Python, so determine where python.exe is installed on your computer. Look for a folder such as c:\Python27 (it may have been installed with ArcGIS). If you cannot find python.exe on your computer, visit python.org [18] and download and install Python 2.7 in the default path.

    In this tutorial, I will assume that c:\Python27\python.exe is your Python path. If your Python is installed in a different location, just adapt the path in the examples to match the path of your own python.exe.
  9. In Windows Explorer, go to My Documents\Mapbox\export and find PhillyBasemap.mbtiles. These are all your tiles. Copy this file to an easy place to remember, such as c:\data\Philadelphia. For some reason, I have not been able to extract tiles in the past when they are sitting in the My Documents folder.
  10. Open a command prompt. (On Windows, click Start and then type cmd in the search box.)
  11. Extract all your tiles by running a command such as the following:
    c:\python27\python.exe c:\mbutil\mb-util c:\data\Philadelphia\PhillyBasemap.mbtiles c:\data\Philadelphia\PhillyBasemap
    Notice that the parameters are, in order, the path to Python, the path to the mb-util utility, the path of your compressed tiles, and the new folder where you want your tiles uncompressed. If any of your paths are different, you will need to adjust them when you type this command. After you run this command, you should see a set of uncompressed tiles in the folder c:\data\Philadelphia\PhillyBasemap.
     Extracted tiles
    Figure 5.20

Hosting and testing the tiles

If you navigate around your folder of unpacked tiles, you'll notice that the images are extracted into a highly organized structure of level\column\row. This structure is understood by various mapping programs and APIs, so all you have to do at this point is put your tiles onto a web-facing server. A convenient place for you to experiment with this is the www folder in your PASS space, a public-facing directory that all Penn State students are given in their personal web space. We will place the tiles there and then test them in a web map.

  1. In Windows Explorer, right-click your folder of uncompressed tiles and click Send to > Compressed (zipped) folder. This should produce a folder titled PhillyBasemap.zip.

    Zipping the folder like this allows you to send all the files to PASS at once, rather than uploading them individually (which would be too tedious to be practical).
  2. Open a web browser to the PASS Explorer site at http://explorer.pass.psu.edu [19].
  3. Log in with your Penn State Access Account and navigate to the [www] folder.

    If you don't have this folder, go to https://www.work.psu.edu [20] and click on a link on the left-hand side for requesting a personal web space. Complete the short required quiz, then wait a minute or two for your personal page to be created.

    Manually creating a folder named [www] will not work for this exercise; you must request this folder as a personal web space from Penn State.

    Regardless of whether your [www] folder has already been created or not, it will be helpful to request a bit more storage. By default, Penn State gives you 100 MB, but you can increase this limit to up to 10 GB. You can do this on the same site https://www.work.psu.edu [20] by adjusting the Change quota dropdown to 10 GB.
     
  4. Within [www] create a new folder called [tiles] and navigate into it.
  5. Click the Upload button.
  6. Browse to your PhillyBasemap.zip folder and check the Auto Extract box.
     PASS upload
    Figure 5.21
  7. Click Upload, and wait for a bit as your tiles are uploaded and extracted. When you are finished, you should be able to navigate to a structure like below:
     Extracted files in PASS
    Figure 5.22
  8. Test out a tile by hitting a URL of the structure http://personal.psu.edu/<your_PSU_ID>/tiles/PhillyBasemap/15/9555/12400.png [21]. You should get back an image.
     Test image from server
    Figure 5.23
    Now, let's see if we can add this to an online map to test the tile appearance and navigation experience. We will use the ArcGIS.com Map Viewer, which is browser-based software for making a web map. It is free to use for experimentation or for sharing maps with the public.

    We're using the ArcGIS.com map viewer here solely because it is a quick and handy way to test a tiled map layer. There is no dependency on ArcGIS software in this walkthrough.
  9. Open a browser to http://www.arcgis.com/home [22] and click the Map link at the top of the screen. You'll see an Esri-provided map, which we will switch out for our Philadelphia basemap.
  10. Click on Modify Map, then Add > Add Layer from Web.
  11. Choose to add data from A Tile Layer, and then populate the dialog box as shown below, substituting your own Penn State ID in the URL, but leaving everything else the same. Note that to set the extent, you must click Set Tile Coverage and draw a box around Philadelphia. You don't need to get the exact same coordinates shown below.
     Add tile layer in ArcGIS.com viewer
    Figure 5.24
  12. Click Add Layer. If nothing happens for a long time, repeat and make sure there is no error in the URL field. Navigate around your Philadelphia map to test it out. Performance should be fast and crisp. If you turn on a web developer utility such as Firebug, you should be able to see the tiles being brought into the map.
  13.  Map tiles in ArcGIS.com
    Figure 5.25
    Optionally, you could add other web layers on top, but, for the sake of time, this will not be described here. In future lessons, you'll learn how to build this type of web map programmatically without using ArcGIS.com.

Lesson 5 assignment: Examine tiled maps, then build your own

Now that you've gone through a walkthrough and built your own tiled map, hopefully your appreciation has increased for the amount of cartographic design and effort required for producing a web basemap. In this week's assignment, you'll look at some existing tiled maps, then get some practice building your own.

Do the following:

  • Find four online maps that use tiled maps in the background. These can be either rasterized image tiles or vector tiles. For each map, comment on where the tiles are coming from and how they were produced, if you can figure it out. It's okay if some of your tile providers are repeated (for example, two of the four might use Google Maps), but try to find at least one web map that uses a "homegrown" tiled basemap in the background. Simple local map viewers produced by city and county GIS websites are good places for this.
  • Create a tiled map of rasterized images using some or all of your term project data. Depending on the nature of your data, this can be a basemap or a map with several thematic layers that is designed to overlay a basemap.

    You should design this map using TileMill and CartoCSS and upload it to your PASS space following the same procedures we used in the walkthrough. The map should be designed using sound cartographic principles, including aesthetically pleasing colors, an intuitive visual hierarchy, and a useful labeling scheme. Take advantage of scale-dependent drawing rules on your layers so that the user is not overwhelmed with information at any particular scale. Zooming into the map should reveal progressively more information.

    Be careful with how many levels of tiles you build for this assignment. To stay on focus during this course, please limit your jobs to under 50,000 tiles at first. After the course, you can experiment with making bigger tilesets if necessary. Also, if your tiled map consists of thematic layers, make sure the tile background is transparent.

Please produce a report containing all the information requested in the first bullet above, as well as a URL to your tileset on your PASS space that I can test in a web map as shown in the walkthrough. Then submit the report to the Lesson 5 drop box on Canvas.


Source URL: https://www.e-education.psu.edu/geog585/node/705

Links
[1] http://blogs.esri.com/esri/arcgis/2010/03/05/measuring-distances-and-areas-when-your-map-uses-the-mercator-projection/
[2] http://msdn.microsoft.com/en-us/library/bb259689.aspx
[3] https://mts0.google.com/vt/lyrs=m@241289412&amp;hl=en&amp;src=app&amp;x=74&amp;y=96&amp;z=8&amp;s=Galile
[4] https://github.com/mapbox/vector-tile-spec
[5] https://www.youtube.com/watch?v=se2cd3BMYRY
[6] https://www.mapbox.com/mapbox-studio/
[7] http://www.azavea.com/blogs/labs/2015/05/converting-mapbox-studio-vector-tiles-to-rasters/
[8] http://blogs.esri.com/esri/arcgis/2015/07/20/vector-tiles-preview/
[9] http://openlayers.org/en/master/examples/mapbox-vector-tiles.html
[10] https://github.com/SpatialServer/Leaflet.MapboxVectorTile
[11] https://github.com/geometalab/Vector-Tiles-Reader-QGIS-Plugin
[12] http://mapnik.org/
[13] https://github.com/mapnik/mapnik/wiki/GettingStartedInXML
[14] https://github.com/mapnik/mapnik/wiki/GettingStartedInPython
[15] https://www.mapbox.com/tilemill/
[16] https://www.mapbox.com/tilemill/docs/guides/metatiles/
[17] https://github.com/mapbox/mbutil
[18] http://www.python.org
[19] http://explorer.pass.psu.edu
[20] https://www.work.psu.edu
[21] http://personal.psu.edu/&lt;your_PSU_ID&gt;/tiles/PhillyBasemap/15/9555/12400.png
[22] http://www.arcgis.com/home