Session 11 - JavaScript, Part 4

Harvard Extension School  
Spring 2021

Course Web Site: https://cscie12.dce.harvard.edu/

Topics

  1. XHR or Ajax
  2. JS Data Structures - array and "object" (key/value pair)
  3. JSON - Creating Content with JS and DOM
  4. JSON - Creating Content with jQuery
  5. Creating Content with XHR, jQuery, and JSON
  6. Same Origin Policy and CORS
  7. Javascript Templates
  8. JavaScript Templates - Example
  9. JavaScript Templates - Apple Picking
  10. JavaScript Templates - Courses
  11. Course Search Example - Live!
  12. Handlebars - A few more notes
  13. Handlebars - Compiled Templates
  14. Maps
  15. Google Maps JavaScript API
  16. Map with Several Markers
  17. Maps - Leaflet JS and Open Street Map
  18. Pancake Menu
  19. Form Price Calculation

Session 11 - JavaScript, Part 4, slide1
XHR or Ajax, slide2
A Collection of Technologies, slide3
What can you do with Ajax/XHR?, slide4
JS Data Structures - array and "object" (key/value pair), slide5
More JSON objects, slide6
JSON - Creating Content with JS and DOM, slide7
JSON - Creating Content with jQuery, slide8
Creating Content with XHR, jQuery, and JSON, slide9
Same Origin Policy and CORS, slide10
Same Origin Policy, slide11
CORS - Cross Origin Resource Sharing, slide12
Javascript Templates, slide13
Better with Templates!, slide14
JavaScript Templates - Example, slide15
JavaScript Templates - Apple Picking, slide16
JavaScript Templates - Courses, slide17
Course Search Example - Live!, slide18
Handlebars - A few more notes, slide19
Handlebars Conditionals, slide20
Handlebars - Compiled Templates, slide21
Maps, slide22
Embed a map created using Google's Tools, slide23
Google Maps JavaScript API, slide24
Placing a Marker, slide25
Marker with an Information Window, slide26
Map with Several Markers, slide27
Maps - Leaflet JS and Open Street Map, slide28
Pancake Menu, slide29
Form Price Calculation, slide30

Presentation contains 30 slides

XHR or Ajax

ajax

XHR/Ajax flow:
web_schematic_ajax.png

  1. The browser makes an HTTP GET request for the search page
  2. The server responds with the HTML content.
    Note that we are not showing browser requests for other page resources such as image, CSS, or JavaScript files
  3. The user types in a search term and clicks the submit button (or presses "enter")
  4. A JavaScript event handler is triggered on the form submit, and JavaScript sends an HTTP request via the browser to the server. This HTTP request contains the search term typed in by the user as a parameter sent to the server
  5. A program on the server searches the course database for any courses that contain the search term.
  6. The program (via the web server) returns the matching courses in a data format to the browser (typically in JSON format)
  7. The browser receives the data, hands it off to the JavaScript, and the JavaScript formats the data into HTML and updates the search area on the page

What the browser does

What the server does

The server is simply responding to HTTP requests, and sending responses back to the browser. The "how" of looking up courses in a database based on a keyword search term is beyond the scope of this lesson -- we will demystify that part later in the course.

A Collection of Technologies

So we see Ajax is really a collection of technologies that are working together to send and receive data to and from the web server based on user actions, and to format and display this data to the user.

Some of the key technologies involved are:

What can you do with Ajax/XHR?

JS Data Structures - array and "object" (key/value pair)

JSON - JavaScript Object Notation

JSON array Example:


         ['Autumn', 'Winter', 'Spring', 'Summer']
       

JSON object Examples:

Simple "name/value" pairs:


         {
           "lastName" : "Bacow",
           "firstName" : "Lawrence",
           "email" : "president@harvard.edu"
         }
       

"name/value" pairs, with values being an array (list) of things:


         {
           "apples" : ['Macoun','Empire','Honey Crisp','Albemarle Pippin'],
           "oranges" : ['Naval Orange','Tangelo','Clementine','Valencia Orange']
         }
       

More about JSON

A data format widely used in Ajax is JSON -- this format is used to pass data from the web server to the browser, and it is in a format that is easily worked with in JavaScript.

We first need to take a look at the JSON format and how we can work with it, and then we'll see how this works together in Ajax.

The introduction to JSON from json.org is a great place to start:

JSON (JavaScript Object Notation) is a lightweight data-interchange format. It is easy for humans to read and write. It is easy for machines to parse and generate. It is based on a subset of the JavaScript Programming Language, Standard ECMA-262 3rd Edition - December 1999. JSON is a text format that is completely language independent but uses conventions that are familiar to programmers of the C-family of languages, including C, C++, C#, Java, JavaScript, Perl, Python, and many others. These properties make JSON an ideal data-interchange language.

JSON is built on two structures:

These are universal data structures. Virtually all modern programming languages support them in one form or another. It makes sense that a data format that is interchangeable with programming languages also be based on these structures.

More JSON objects

In short, a JSON object is an unordered list of name/value pairs.

A JSON object example:

Note that the object (collection of name/value pairs) is enclosed in {curly braces}.
The name/value pair has a colon after the name (i.e. name: value), and the name/value pairs are separated by a comma.

So the object above represents the following:

NameValue
lastNameBacow
firstNameLawrence
emailpresident@harvard.edu

JSON Array

In short, a JSON array is an ordered sequence of values. Values are strings (in quotes), numbers, arrays, objects, or boolean (true/false) values.

A JSON array example

Strings, Objects, and Arrays can all be values!

In the examples above, we've shown the values as strings (e.g. "Spring" or "Bacow"). But values can also be arrays or objects. Below is a more complex JSON object that includes two name/value pairs whose values are an array of strings ("seasons") and the other an array of objects ("days"):

There are several online JSON viewers and validators you can use that are helpful for exploring, writing, or validating JSON structures:

JSON - Creating Content with JS and DOM

JSON - Creating Content with jQuery

Now that we have a sense of JSON and its "object" and "array" structures, let's see how we can create content from JSON using JavaScript.

We'll use the jQuery JavaScript library to build up markup based on data in the JSON structure, and then we'll place it on the page.

We'll use the "seasons" array to build an unordered list of seasons.

The JSON data:

The result we will build:
json-create-content-1.png

Let's take a look at the important parts:

HTML:

JavaScript:

All Together

Complete markup is:

Creating Content with XHR, jQuery, and JSON

So now we'll bring Ajax into the picture by requesting data from the server.  The server will return the seasons JSON data, and we will then pass array of seasons to our function that will create the unordered list.

The jQuery library makes working with Ajax requests very easy.  jQuery has a getJSON method that makes requesting and receiving JSON data from the server easy. In the simplest case, we'll pass two parameters when we use the getJSON method: a URL that will be requested, and a "callback" function that will be called on a successful retrieval of JSON data. A simple case where we make a JSON request to the url "seasons.json" and simply log the data to the JS console looks like:

$(document).ready(function(){
  $.getJSON("seasons.json",function(data) {
    console.log(data);
  });
});

All together

So we take our previous example, where we built a list from a JSON data structure, and change it so that the JSON data comes from an Ajax request (using the getJSON) method of jQuery:

Same Origin Policy and CORS

When working with XHR/Ajax, it is important to understand some basic concepts of web application security. Some concepts we'll briefly describe are:

These are important to understand since the browser restricts the XHR calls allowed.

Same Origin Policy

This is the simplest policy, and will cover all cases where the data you are requesting comes from the same hostname as the page making the request.

Under the same origin policy, a web browser will permit JavaScript contained in a web page to access data from the same origin -- or the same "hostname" of the URL.

So if your JavaScript and the data it is accessing all come from the same origin (hostname in the URL), the browser will permit this communication.

If the JavaScript and the data come from different origins (hostnames), then the browser will block this for security reasons.


For JavaScript to access data that comes from different origins, you'll need to make sure CORS headers are set correctly.

CORS - Cross Origin Resource Sharing

CORS is a common and preferred method for handling cross-origin requests.

Using CORS, it is possible for the browser and server to determine whether or not to allow a cross-origin request. This happens as part of the HTTP request. The server can specify access control rules, which a browser will then use to determine whether a cross-origin request is permitted.

We won't go into how make the server-side CORS compatible right now. From the front-end perspective, it is enough to understand that you are using a CORS-enabled service. You don't need to do anything to enable CORS from within your JavaScript -- it already is there and is used by the browser.

HTTP response header:
access-control-allow-origin: *

For example, the service from "cdn.rawgit.com" (e.g. apples.json) uses CORS, which is why the "Pick Your Own Apples" example worked fine, despite the different origins between the page the JSON.

MDN: Cross-Origin Resource Sharing

Javascript Templates

Having the separation of data coming from the server-side in the form of JSON (or even some other format), and doing the work to display and present it on the client-side is a beneficial separation of concerns.  If the formatting and display requirements change, it is a change to the front-end (client-side) code (HTML, CSS, JS) as opposed to the server-side.  However, while the technique we've seen to process JSON into markup and place it on the page is entirely on the client-side (as opposed to server-side), a big drawback is that we've done this entirely in JavaScript.  

For example, to build a list of seasons, we did something like:

Good News:  We do have separation between front-end and back-end!  Front-end does the structuring and display; the back-end provides the data.

Bad News:  We have mixing of concerns between structure (markup) and function (JavaScript).  We do not have good separation between structure (HTML), style (CSS), and function (JS) within the front-end.  We have to read through the JavaScript to figure out how our data will be structured.  If we want to change how the data is structured, we need to edit the JavaScript. 

Better with Templates!

Using JavaScript templates, building a list could look something like:

There are a variety of libraries for making templates in JavaScript.  I'll mention four here, and we'll focus on one in our examples (Handlebars):

KISS - Keep It Simple.
Before we get started with an example, I want to put a strong recommendation to follow the keep your templates simple principle.  Trimpath JST and Mustache forces you to do this -- with Handlebars, you can keep your templates simple, but you can make them complex. Remember that the prime benefit we want to achieve with templates is for our presentation format to be defined separately from our JavaScript code. The more complex our templates get, the further we get from achieving that benefit.

JavaScript Templates - Example

We'll focus on using the Handlebars template library.  Other template systems will do similar things though, with different syntax.

With Handlebars, the key parts are:

Handlebars syntax are based on double curly braces, like {{ handlebar expression }}

So to iterate through the <code>seasons</code> array, our Handlebars template would look like:

<ul>
  {{#seasons}}
  <li>{{.}}</li>
  {{/seasons}}
</ul>

To re-implement our "seasons list" example using Handlebars, we could something like:  

See the working examples:

JavaScript Templates - Apple Picking

PYO apples

JSON Data

{
    "apples": {
        "picking_now": [
            "Macoun",
            "Empire",
            "Cortland",
            "McIntosh",
            "Honey Crisp",
            "Gala",
            "Red Delicious"
        ],
        "picking_soon": [
            "Golden Delicious",
            "Northern Spy",
            "Baldwin",
            "Mutsu"
        ],
        "picking_done": [
            "Ginger Gold",
            "Jonamac",
            "Gravenstein"
        ]
    }
}

Handlebars Template

<div id="appleslisttemplate" class="handlebars">
  <h3>Picking Now</h3>
  <ul>
    {{#apples.picking_now}}
      <li>{{.}}</li>
    {{/apples.picking_now}}
  </ul>
  <h3>Picking Soon</h3>
  <ul>
    {{#apples.picking_soon}}
      <li>{{.}}</li>
    {{/apples.picking_soon}}
  </ul>
  <h3>Missed Out for This Year</h3>
<ul>
    {{#apples.picking_done}}
    <li>{{.}}</li>
    {{/apples.picking_done}}
</ul>
</div>

JavaScript

let dataUrl = 'https://cdn.rawgit.com/dpheitmeyer/b9fe0838e27b28db1287/raw/38356d6675e355215b7bf22ff75c520a8fbe8850/apples.json'

$(document).ready(function(){
  $.getJSON(dataUrl,
    function (data) {
      var mysource = $('#appleslisttemplate').html();
      var mytemplate = Handlebars.compile(mysource);
      var myresult = mytemplate(data)
      $('#appleslist').html(myresult);
  });
})

JavaScript Templates - Courses

Let's take a look at a Handlebars template that processes information about Courses from the Faculty of Arts & Sciences at Harvard University.

Courses - ExampleCourses - JSFiddle

Course data. First, here is a JSON string built based on a query of the course database for all the courses with a keyword of "food".  It would be typical to have this query built from a form and return "live" results in JSON, but to keep things simpler for this example, we'll just use this static list:  courses.json

Handlebars template. In this JSON structure, there is an array of course objects (the "name" part is courses, the value is the array of course objects). In Mustache, we'll need to iterate through the courses, and for each one, we'll output some information about the course. Assuming we pass the Mustache template the JSON object above, this would look something like:

<div id="courselisttemplate" class="handlebars">
<ul>
{{#courses}}
    <li class="course">
    <span class="number">{{course_group}} {{num_int}}{{num_char}}</span>
    <br/>
    <span class="title">{{title}}</span>
    </li>
{{/courses}}
</ul>
</div>

Notice how we can simply reference the "name" of a name/value pair in an object, like {{title}}.

The above template would render like:
course_list1.png


We'll make this example a bit more complete by adding the faculty_text and meeting_text to the output. Also, we can add the number of courses returned by using {{courses.length}}.

<div id="courselisttemplate" class="handlebars">
<p>{{courses.length}} courses were found</p>
<ul>
{{#courses}}
    <li class="course">
    <span class="number">{{course_group}} {{num_int}}{{num_char}}</span>
    <br/>
    <span class="title">{{title}}</span>
    <br/>
    <span class="faculty">{{faculty_text}}</span>
    <br/>
    <span class="meeting">{{meeting_text}}</span>
    </li>
{{/courses}}
</ul>
</div>

And the result would look like:
course list

Course Search Example - Live!

Data

Markup (Form Part)

<form id="course_search_form">
  <div>
      <input type="text" id="search_query" name="search_query" size="40" placeholder="Search by title, instructor, description" />
      <input type="submit" name="submit" value="Search" />
  </div>
  <div id="courselist">
      Course Listings will be updated here.
  </div>
</form>

JavaScript

$(document).ready(function () {
    console.log("DOM READY!");
    var course_template_source = $('#courselisttemplate').html();
    var course_template_compiled = Handlebars.compile(course_template_source);

    console.log(course_template_source);
    console.log(course_template_compiled);

    $('#course_search_form').submit(function (e) {
        console.log("form submitted!")
        var search_term = $('#search_query').val();
        console.log("search term is: " + search_term);
        $.getJSON('/ajax/course_search', {
                'keyword': search_term,
                'output': 'json'
            },
            function (data) {
                console.log("data retrieved is: ");
                console.log(data);
                var myresult = course_template_compiled(data);
                $('#courselist').html(myresult);
            }
        )
        e.preventDefault();
    });
});

Handlebars Template

<div id="courselisttemplate" class="handlebars">
<p>{{courses.length}} courses were found</p>
<ul>
    {{#courses}}
    <li class="course">
        <span class="number">{{course_group}} {{num_int}}{{num_char}}</span>
        <br/>
        <span class="title">{{title}}</span>
        <br/>
        <span class="faculty">{{faculty_text}}</span>
        <br/>
        <span class="meeting">{{meeting_text}}</span>
    </li>
    {{/courses}}
</ul>
</div>

Handlebars - A few more notes

See: JSFiddle Example of Handlebars Loop Techniques

With Handlebars, you can also iterate explicitly with {{#each }}

With the context of our "seasons" JSON:

{
  "seasons": [
    "Spring",
    "Summer",
    "Autumn",
    "Winter"
  ]
}

We could do:

<ul>
    {{#each seasons}}
    <li>{{.}}</li>
    {{/each}}
</ul>

We can also specify the name of the parameter to be used within the loop:

<ul>
    {{#each seasons as |season|}}
    <li>{{season}}</li>
    {{/each}}
</ul>

Both of the above would produce:
seasons-1.png

If needed, you can also pass in the index position as well (remember that the first item in an array is index position = 0, second item is index position = 1, etc.).

<ul>
    {{#each seasons as |season pos|}}
    <li>{{pos}} {{season}}</li>
    {{/each}}
</ul>

Which would produce:
seasons1.png

Handlebars Conditionals

Sometimes we might want to format data based on certain conditions. For example, if an email address or URL exists for a person, we might want to include a mailto: link or a link to the URL. Handlebar conditionals allow to do that.

People in JSON:

{
    "people": [
        {
            "name": "Lawrence Bacow",
            "url": "http://president.harvard.edu/"
        },
        {
            "name": "Alan Garber",
            "url": "http://provost.harvard.edu/people/alan-m-garber"
        },
        {
            "name": "Katie Lapp"
        }
    ]
};

For example, if we wanted to hyperlink a name if a URL exists, our Handlebars template could look something like:

<ul>
{{#each people as |person|}}
  <li>
  {{#if person.url }}
    <a href="{{person.url}}">{{person.name}}</a>
  {{else}}
    {{person.name}}
  {{/if}}
  </li>
{{/each}}
</ul>

And this would produce:
people1-1.png

Handlebars - Compiled Templates

It is possible to precompile your Handlebars template (the result is a JavaScript function), and then include the compiled templates via a script element.

Advantages of precompiling the templates include:

So here's an example where I've compiled the course list template into courseslist.js. Now there's no compile step, just a reference to Handlebars.templates.courseslist to access the template function.  Note too that we are using the "runtime" version of the Handlebars library - handlebars.runtime.min.js 

Note the command to compile the template looked like:

$ handlebars.js courseslist -f courseslist.js

where courseslist is the source template, and courseslist.js is the generated compiled JS file

Maps

With JavaScript APIs for Google Maps, Bing Maps, and MapQuest, and Open Street Maps (Overpass API), it is very easy to embed customized maps on your website.

We'll take a look at a few different examples of how to include maps on our site:

Each approach has some advantages and shortcomings.  Essentially the trade-offs will be increased complexity with increased flexibility and control.

Embed a map created using Google's Tools

You can create a map using Google's tools, and then simply click the cog icon and select "Share or embed". The "embed map" option will give you an iframe code you can use in your web page. It doesn't get much simpler than this. If you want to embed a map with the location of a single store or organization, then this technique will work well.

A map I created for the John Harvard Statue:

map2.png

The iframe code provided by Google:

<iframe style="border: 0;"
  src="https://www.google.com/maps/embed?pb=!1m14!1m8!1m3!1d1473.7541883076933!2d-71.11691195715973!3d42.37431548010373!3m2!1i1024!2i768!4f13.1!3m3!1m2!1s0x0%3A0x1b5299f9d1edac24!2sJohn+Harvard+Statue!5e0!3m2!1sen!2sus!4v1428381157098"
  width="600" height="450"></iframe>

And the map that is embedded:

For further information about the Google Map Embed API, please see
https://developers.google.com/maps/documentation/embed/guide

Cases where the embed approach wouldn't work so well:

An example where a static embedded map would not work is with the Student Locations tool for this course.  This tool allows students to enter their location information, and then the map is built dynamically based on this data.  In addition, this tool displays different data, depending on whether the viewer is a student or an instructor.  This is a case where we'll need more control over the map than a straightforward embed approach will allow.

The Google Maps JavaScript API will give us that control.

Google Maps JavaScript API

There is extensive documentation for the Google Maps JavaScript API, at
https://developers.google.com/maps/documentation/javascript/tutorial that you can consult. But for now, it will be enough to walk through some examples.

We'll learn by example with a

Basic Map

So first, we'll start with a map that is centered on the John Harvard statue in Harvard Yard.

To use the Google Maps JavaScript API, we'll need to load the Google Map JavaScript via a script element, and then we'll need to configure the map and then place it on the page.

Load the JavaScript:

<script src="https://maps.googleapis.com/maps/api/js?v=3.exp"> </script>

Use the JavaScript:

var map;
/* John Harvard Statue
   42.374474,-71.117207 */
var mylat = 42.374474;
var mylng = -71.117207;
function initialize_map() {
    var mapOptions = {
        zoom: 17,
        center: new google.maps.LatLng(mylat, mylng)
    };
    map = new google.maps.Map(document.getElementById('map-canvas'),
    mapOptions);
}
google.maps.event.addDomListener(window, 'load', initialize_map);

Full example:

Basic Map working examples:

Placing a Marker

map4.png
To put a marker on the map, we'll create a google.maps.Marker object. When we create the marker, one of the parameters we'll pass in as part of the configuration data structure is the map we want to place it on.

So if we've created a google.maps.Map object as the variable harvard_yard_map and a googlemaps.LatLng object as the variable latlng_statue, the new JavaScript code to create a marker for the John Harvard statue would look like:

var marker = new google.maps.Marker(
  {
    position: latlng_statue,
    map: harvard_yard_map,
    title: 'John Harvard Statue'
  }
);

Note there is a single parameter passed to the google.maps.Marker method -- this parameter is a JavaScript data structure of name/value pairs.

Full example:

Map with Marker working examples:

Read more about Markers, Google JavaScript API documentation for additional information, such as using customized icons.

Marker with an Information Window

Now that we have a marker on the map, we can add an information window.
map3-1.png

There are three basic things to pay attention to:

These steps are shown below:

/* Create content.  Here we are concatenating strings together with the JS "+" operator */
var statue_info = "<strong>John Harvard Statue</strong><br/>" +
    "<a href='http://www.summer.harvard.edu/blog-news-events/3-lies-harvard'>3 Lies of Harvard</a>";

/* Create the InfoWindow object; note we pass in the content */
var statue_infowindow = new google.maps.InfoWindow({
    content: statue_info
});

/* Create the marker (we've seen this before) */
var statue_marker = new google.maps.Marker({
  position: statue_latlng,
  map: harvard_yard_map,
  title: 'John Harvard Statue'
});

/* Create the listener so that the info window opens when the marker is clicked */
google.maps.event.addListener(statue_marker, 'click', function() {
  statue_infowindow.open(harvard_yard_map,statue_marker);
});

Full example:

Map with Marker working examples:

Map with Several Markers

Now let's use JSON data to add several markers to a page.  

Map with Ivy Schools from JSON Data - Live Example

The JSON data (schools.json) we'll use is a list of schools in the Ivy League -- which contains a latitude, longitude, name, and URL.

We'll use jQuery to get the JSON data, and we'll use the Google Maps API to create the map. The full code is shown below, but the key part to creating the markers is where iterate through the list of schools:

 for (var i in school_list) {
  var school = school_list[i];
  var mylatlng = new google.maps.LatLng(school.lat, school.lng);
  var marker = new google.maps.Marker({
                                       position: mylatlng,
                                       map: ivy_map,
                                       title: "school.name"
                                      });
}

map

Full code:

Maps - Leaflet JS and Open Street Map

Leaflet: an open-source JavaScript library for mobile-friendly interactive maps

Pancake Menu

Mobile Pancake/Hamburger Menus

pancake menu

pancake menu

Form Price Calculation

Let's revisit the Dado Lunch Form, this time calculating a price!

lunch menu