Published on GEOG 865: Cloud GIS (https://www.e-education.psu.edu/geog865)

Home > Lessons > Lesson 8: GIS as a service using ArcGIS Online

Lesson 8: GIS as a service using ArcGIS Online

Overview

So far in our exploration of software as a service (SaaS) providers, we have focused largely on map design and construction. We’ve also seen how datasets can be uploaded and stored on the cloud. In this lesson, we’ll move forward and look at how GIS tools and algorithms can be invoked in a SaaS environment.

You got a taste of GIS as a service back when you used CARTO to aggregate farm dropoff points to neighborhoods. This required an algorithm to run determining the neighborhood where each point was located. The neighborhoods layer was then updated with a field showing the count of all points inside. If this were run locally, it would require you to install GIS or other spatial data processing software. Offloading this operation to the cloud requires you to solely focus on the input and output data.

Many other GIS operations are possible in the cloud; all that’s needed are some known input/output formats and some server logic that can then process the data. A popular input/output format is vector features. You’ve seen how there are lots of known formats for that, such as GeoJSON, CSV, KML, etc. Once the server receives these, it can perform operations such as buffering, intersection, routing, drive time analysis, etc., and send back the result in the form of more vectors, an image, or perhaps even textual reports. These analyses might incorporate sophisticated datasets from the cloud provider, such as road networks, address databases, or demographic information. Cloud providers can charge a metered fee, deducting money or credits for each operation performed, or they can charge flat monthly fees for different tiers of capabilities.

Although Esri is not the only company that offers GIS operations as a cloud service, it is clearly an area where they specialize. Esri ArcGIS Desktop software has hundreds of tools running all kinds of GIS operations. The challenge for Esri (and other cloud service providers) is to expose these kinds of tools online through an interface that’s intuitive to people who may have never used any GIS before. These users may know exactly what they want to accomplish, but would not be familiar with GIS terms like clip, union, buffer, etc. Companies offering GIS as a service must clearly define these terms or simplify them. Pause and spend a few minutes looking over the Perform analysis [1] page to see how Esri uses a combination of graphical icons and simplified terms to explain the spatial analysis capabilities in ArcGIS Online.

In this lesson, you'll use GIS services on ArcGIS Online to derive service areas, join demographic variables to those, and export data for further analysis outside the cloud. These are just a few of the many possible operations offered by ArcGIS Online, but they should give you a taste of how to invoke the analysis and manage Esri service credits.

Lesson Objectives

At the successful completion of this lesson, you should be able to:

  • understand how spatial analysis tools can be exposed through software as a service (SaaS); and
  • use analysis tools in ArcGIS Online to solve geographic problems.

Deliverables

  • Become familiar with Final Term Project Requirements and Rubric 
  • Submit: L08: Final Project Abstract (summary) of Project Idea
  • Complete: L08: Assignment

Preparing for this lesson's walkthrough exercises

Let’s get some practice with ArcGIS Online GIS analysis services. Imagine you’re working for a sushi delivery company that has made its way to fame via an app accessed from people’s smartphones. Customers use the app to order fresh-made sushi to be delivered to their home. Your company makes the sushi in small “stores” (similar to pizza delivery outlets) and delivers it from those locations.

Unfortunately, business isn’t doing too well, and the company has been forced to cut its number of stores. You are tasked with determining one of the four stores to shut down in the Yakima metro area. You’ve just learned about some geographic analysis that you could perform online to help you with your decision. You already know that you want to consider the area your delivery cars can reasonably reach from each store. You also want to find out about the customer base of each store, including how many people live near each store, how much they tend to spend on restaurants, and how many own smartphones.

The only data you have at this point is a spreadsheet containing the locations of your stores and the amount you pay in rent each month for the commercial space. Let’s start by getting that data into ArcGIS Online.

First download the data for this lesson [2]. Then do the following:

Selecting symbol for an uploaded ArcGIS Online layer
Figure 8.1: Selecting symbol for an uploaded ArcGIS Online layer
  1. Create a new empty map.
  2. Click the button to Open in Map Viewer Classic.
  3. Click Add > Add Layer From File. (If you don't see this option, click Modify Map.)
  4. Open delivery.csv in a spreadsheet program like Microsoft Excel. Examine the data, but then close it out without saving it as any other format. It needs to remain a .csv instead of an .xlsx.
  5. Log in to ArcGIS Online with your Penn State credentials as you did earlier.
  6. Browse to delivery.csv.
  7. Click Import Layer, and examine the resulting dialog box.
  8. Notice that the software is guessing that it should use the ADDRESS field to derive the location. This is what we want. (Be sure that United States is selected in the dropdown.)
  9. Click Add Layer. You should see four dots appear on the map in the greater Yakima area.
  10. Change the style of the dots to be Single Symbol. Click the blue Options button that appears, and adjust the symbol style to look how you want. If you click Symbols (next to the black dot), you’ll be presented with even more choices.
  11. Save the symbology settings, and then save your map with a name like Delivery analysis.

Deriving service area polygons

The first thing you want to do is find out what area is served by each store. The company has learned that customers demand their sushi within 20 minutes of ordering it. Twelve minutes are typically required to make the food, and eight minutes are allotted for delivery. Let’s find out the areas that lie within an eight-minute drive of each store.

  1. In the ArcGIS.com map viewer, open your Delivery analysis map from the previous section of the lesson.
  2. Click the Analysis button, which is up on the top menu next to the basemap selector.
  3. Expand Use Proximity and click Create Drive-Time Areas.
  4. Change the drive time to 8 minutes, and leave all the other default settings. Notice, however, that there are lots of options for taking into account live and historical traffic. With all this information available on the back end, you can imagine developers writing apps where a route could be calculated using current traffic conditions.
  5. Click Run Analysis, and wait for a minute for the result to be calculated. You should eventually see some irregularly-shaped polygons appear around each store, showing the area that can be reached within an 8 minute drive of each. These “service area” polygons were not derived from any street data we uploaded, nor did they come from the basemap, which is just an image. They did come from detailed street network data that Esri has assembled for use by its cloud services.
  6. Change the symbol of the drive time polygons to be about 35% transparent. Note that the layer itself is already 50% transparent, but we want to apply transparency on the individual polygons so that we can better see where there is overlap.
     
    Adjusting symbol transparency in ArcGIS Online
    Figure 8.1: Adjusting symbol transparency in ArcGIS Online
    Another way to get a feel for the overlap is by selecting individual polygons. You can do this from the attribute table as described in the next steps.
  7. Click the Show Table icon that appears if you hover your mouse underneath the Travel from delivery layer.
  8. Click a row in the table to see the corresponding polygon highlighted on the map. This table is also useful because it shows you the number of square kilometers covered by each polygon.
  9. Look through the table and the map, and consider the following questions: Which store covers the smallest amount of area, and how many square kilometers is this area? Which store covers the largest area? Where is there a lot of overlap between store coverage? Where are there gaps?
  10. Save your map.

    Note that creating service area polygons in ArcGIS Online is not free. Your ArcGIS Online account was just charged some credits for using this service. Fortunately, the Penn State ArcGIS Online organization, of which we are members, is providing the credits to us and we don't need to worry about it. In your own production setting, however, it's good practice to be aware of how many credits an operation will consume before you run it. To learn how many credits all the different operations on ArcGIS Online cost, take a look at the Esri service credits [3] page. At the time of this writing, it says that it costs 0.5 credits per drive time. In the next part of the lesson we will use Data Enrichment, which is listed as 10 credits per 1,000 attributes (data variables).

The service area polygons we calculated were interesting and somewhat useful, but the raw area of the polygon alone is not enough to help us get a feel for the underlying population served. Parts of the city are much more densely populated than others. Also, people in some neighborhoods tend to eat out more than others. Some neighborhoods might also have a higher density of smartphone usage where people would be inclined to order using your app. We’ll explore these variables in the next section of the lesson.

Joining demographic data with service areas (Data Enrichment)

In this part of the walkthrough, we’ll try to learn a bit more about the customer base that lives within each 8-minute service area polygon we derived. We’ll accomplish this using what Esri calls “Data Enrichment”, in other words, joining and summarizing attributes from extensive demographic databases.

  1. In the ArcGIS.com map viewer, open your Delivery analysis map from the previous section of the lesson.
  2. Click Analysis, and choose Data Enrichment > Enrich Layer.
  3. Be sure your service areas polygon layer is chosen in the drop-down list.
  4. Click Select Variables, and make sure that United States is selected from the dropdown.
    Here’s where you will be tempted to go crazy and add all kinds of interesting demographic and consumer information to your shopping cart of variables. Go ahead and add the following:
     - Total Population (under Population)
     - Total Housing Units (under Housing)
     - Meals at Restaurants  (under Spending)
     - Have a Smartphone (under Behaviors)
     - One other variable you think would be useful for our analysis
  5. Ensure that you have just a handful of selected variables (in other words, make sure you didn’t accidentally click a whole category and get tons of variables added to your cart).
  6. Click Apply. This will close the variable selection dialog box.
  7. In the left menu of your viewer, scroll down and examine the other options for enrichment, but don’t change any of them. Then click Run Analysis and wait a few minutes for the enrichment to occur.
    When the enrichment finishes, you’ll see a service area polygon layer just like the one you already have. The difference is in the attributes.
  8. Open the attribute table for the Enriched travel time layer.
  9. Scroll over until you see some of the new attributes, such as Total Population.
     
    Notice that some of the fields are more useful than others in their current state. We have a smartphone figure that isn’t normalized by anything. We could at least divide that by population to get some kind of an index of smartphone ownership, although it wouldn’t be perfect.
  10. In your attribute table, click the Add Field menu item using the icon at the upper-right of the table.
     
  11. Add a field called PctSmartphone of type Double. (You may need to click the small plus icon to the right of the table to be sure all fields are visible.)
  12. After the PctSmartphone field is added to your table, click the field heading, and click Calculate using the SQL Option.
  13. Calculate the field as MP27002a_B/TOTPOP_CY as shown below. If the table goes blank after you calculate it, click onto another table and then come back to this one.
     
    Calculating a field in ArcGIS Online
    Figure 8.8: Calculating a field in ArcGIS Online
     
  14. Examine your new index of smartphone ownership to see which service areas have the highest ownership.
  15. Calculate another field called TotalRestaurantSpending that represents the monthly restaurant spending multiplied by the number of housing units (NOT population). This gives some idea of the amount spent on restaurants in each service area each month.
  16. Look over all the attributes and think about questions like the following: Did the service area with the largest area also have the largest population? Is there any relationship between smartphone ownership and restaurant spending?
  17. Save your map.

Exporting data

Since your boss only speaks Excel, it might be nice to get your stores spreadsheet back with all these enriching variables added. This can be accomplished via a simple table join from the enriched service areas back onto the delivery points.

  1. In the ArcGIS.com map viewer, open your Delivery analysis map from the previous section of the lesson.
  2. Click Analysis, and click Summarize Data > Join Features.
  3. Set up the join with delivery as the target layer and the Enriched travel time polygons as the join layer. Using the Choose the fields to match button, make sure you are doing an attribute join between the NAME and the NAME_1 fields (do not use “Name”--that was a field added by Esri which caused some clunkiness here with our joins). Take the defaults for all attributes not shown below, and click Run Analysis.

    It may not always be a safe choice to do a join on a name field if there is the possibility of repeated names. In this case, it is fine because the name field contains a unique store number.

    You might have wondered why we didn't do a spatial join. This is because a store location could conceivably fall within two or more service areas. Doing an attribute join ensures a one-to-one relationship between a store and its polygon.
  4. Ensure that a join layer appears symbolized by points. You should be able to open the attribute table on this layer and see all your enriched variables.
    Let’s make a map with this layer. How about total restaurant spending of each service area symbolized by proportionally sized circles?
  5. In the left-hand layer list, hover over the layer name of Join Features to delivery and click Change Style (it’s the icon right next to the attribute table icon).
  6.  Choose to symbolize the TotalRestaurantSpending attribute, and choose Counts and Amounts (Size). Then click its blue Options button.
  7. Play around with the symbol properties until you find something you like.
  8. Turn off the other layers, and examine your resulting map of restaurant spending within each store’s service area.

    Remember that this spending pertains to all restaurants; the amount of spending on sushi at your company's stores could follow a quite different pattern. A choropleth map of the service area polygons might be a be an alternative choice, but it would have its own challenges due to polygons overlapping.

    That’s enough mapping for the time being. Let’s get back to exporting this data to a spreadsheet.
  9. Click Analysis, and choose Manage Data > Extract Data.
  10. Choose to extract Join features to delivery, and set the Study Area to be the same as display (just make sure your map is zoomed out to show all four stores).
  11. Leave the output format as CSV, and choose an output file name.
  12. Click Run Analysis.
  13. In the upper left corner of your ArcGIS.com map viewer, click Home > Content.
  14. Find your extracted CSV, and click on its name to see its item details. (If it's not there, wait a few minutes and refresh.)
  15. Click the Download button to save the CSV to your local computer.
  16. Open the spreadsheet in a program such as Excel, and verify that it contains all the original information about your four stores, plus the enriched variables. As a bonus, you’ll notice the data also comes with Lat and Lon fields since it was geocoded by your software.

Assignment: Making a decision based on your analysis

For this week's assignment, please create a single document containing all of the following:

1.  Unfortunately, there is no button on any GIS, cloud-based or otherwise, that says “Give me the answer”. All the same, you were able to use ArcGIS Online services to learn quite a bit more about the potential customers of your stores. Given what you learned, make a decision about which store should be cut that would minimize the overall financial impact on the franchise.

Write a justification for your manager of about 500 words detailing your decision. This should contain evidence using the enriched variables you derived and any maps you want to make with ArcGIS Online. Discuss the impact and usefulness (or lack thereof) of each variable. If you’re at a loss about what else to include in your report, try adding a map of what the service areas would look like with your selected store cut out.

There is no “right” answer to this question (although there may be questionable or unsupported answers). I am mainly looking for evidence that you’ve thought about the data and the analysis performed in the walkthroughs, and that you can use the output to address a spatial problem.

As you perform any additional analysis and make your maps, keep an eye on your credit usage. You want to leave enough ArcGIS Online credits in your account for your final project (if you are going to use ArcGIS Online in that project).

2. In this lesson, you observed how Esri has tried to put a very user-friendly face on some complex analysis tools in order to make them approachable to people without formal GIS training. What is gained and/or lost under this approach? Are there dangers that the tools might be misused if they are overly "dumbed down", or is the simplification of the tools helpful for everyone?

Study at least two (2) of the ArcGIS Online analysis operations and find their corresponding tools in ArcToolbox. Paste screenshots of both in your report here, and provide some commentary on (1) how the user interfaces have been changed for an ArcGIS Online audience, and (2) how their user interfaces are helped or hindered through this simplification.

Term project overview and abstract assignment

The final week of this course will be dedicated to a term project that each of you will complete to integrate and apply your understanding of Cloud and Server GIS in the context of an application scenario you choose. You will select ONE project option from the list below and submit an abstract in week 8 describing your project idea. To a large degree, you will have the freedom to shape the specifics of your term project around a cloud GIS project that interests you. I hope that this allows you to either focus on a topic related to your day-to-day work or choose an area that sparks your curiosity.

Term Project Options

Here are the options you have for your term project. You should choose ONE of these options:

OPTION 1: Set up an ArcGIS Server-based website using your own data, using EC2 as a hosting service.

OPTION 2: Solve a GIS problem using multiple cloud machines.

OPTION 3: Use ArcGIS Online, Carto, or Mapbox technology to solve geographic data handling problems.

OPTION 4: Design a cloud-based infrastructure that complements an existing GIS. Since this option will deliver a design rather than an example of a working system, the written component will need to be much larger than under the other options.

OPTION 5: Develop your own topic using Cloud GIS. You will need to receive approval from your instructor.

Deliverables

The term project includes the following deliverables:

  • Lesson 8: Final Project Abstract (summary) of Project Idea
    • 1 paragraph project abstract indicating which project option you chose and describing in general terms what you will do and what the budget is for your project. This is due at the end of Week 8. (10 points)
  • Lesson 10 Final Project Video Demonstration
    • 5 minute video demonstration of Term Project, due at the end of Week 10. (30 points)
  • Lesson 10: Final Term Project Report
    • 500 - 1000 word final term project report, due at the end of Week 10. (60 points)

Please use the Term Project Rubric [4]as a guide for implementing your project. It describes all the pieces that need to be present in order to earn an A, such as discussions of cost and security considerations.

Abstract submission

By the end of week 8, you need to submit an abstract (summary) of your project idea. Submit this to the corresponding drop box on Canvas in the form of a document of 200 - 300 words. The instructor will review immediately and post any concerns or needed modifications.


Source URL:https://www.e-education.psu.edu/geog865/node/337

Links
[1] http://doc.arcgis.com/en/arcgis-online/analyze/perform-analysis.htm [2] https://www.e-education.psu.edu/geog865/sites/www.e-education.psu.edu.geog865/files/data/arcgis_online_gis_lesson_data.zip [3] http://www.esri.com/software/arcgis/arcgisonline/credits [4] https://www.e-education.psu.edu/geog865/sites/www.e-education.psu.edu.geog865/files/data/Geog%20865%20Term%20Project%20Rubric.docx