ECDN historical usage reports

You can retrieve ECDN usage data for any duration in the past two years. You may submit your requests using REST APIs or via the web portal. All requests are handled asynchronously. Submissions are immediately acknowledged. Processing is queued up. When ready, you will get an email notification with a link to download the raw data file for the specified duration. Two formats are supported: JSON (coming soon) or CSV.

Submit report requests via the web portal

 Click on "REPORTS" in the page header to bring up the "Request report" page.

request report page screenshot

Steps:

  1.  Specify the data range for the duration of interest. You may enter custom values or select a preset from the dropdown. Default is "Today":
    request report page date range dropdown screenshot
  2. Specify the granularity of aggregating the usage data.
    • For date range duration of <= 24 hours, you may pick either Minute or Hour as the aggregation interval.
    • For date range duration > 24 hours, you can only pick Hour as the aggregation interval.
  3. Specify whether you want data to be further aggregated by "Locations" (default), or you want to the data for all ECDN "Servers".
  4. Specify "All metrics" (default) or pick specific ones from the dropdown:
    request report page metrics dropdown screenshot
  5. Specify the format in which data should be returned: "CSV" - comma separated values (default) or "JSON".
  6. Specify the email address to which the report should be sent when ready.
  7. Click the "Request report" button to submit your request.

Your submission will be acknowledged and uniquely identified with a request id.

request report submission acknowledgement screenshot

All report requests are processed asynchronously. When ready, and email will be sent to the specified address, which has a link to download the report. See below for example reports and how to interpret the data.

Submit report requests via REST API

This is a two step process:

  1. Get an authentication token and store it in a cookie.
  2. Submit request using the cookie for authentication.

This article has examples that use the command line tool curl to make the REST API calls. You may also use any other REST tool of your choice.

Authentication

The API uses session based authentication. To get a valid session, send a POST request to /login API giving your email, and password as input parameters. Note, these are the same credentials you use to login to the ECDN web portal.

$ curl -X POST -d 'email=yourEmailAddress&password=yourPassword' -H "Content-Type: application/x-www-form-urlencoded" https://api.deepcaching.com/login -c cookie.txt
{ "success": true }

Replace yourEmailAddress and yourPassword with your email address and password before making the call.

If successful, you will see {"success":true} response, and the API authentication token will be stored in the file cookie.txt in your local directory.

For the ECDN beta environment use the API endpoint: https://beta-api.deepcaching.com/login

Submit request

The report request can be submitted by making a POST request to the /reports API as shown below. This call will retrieve all usage metrics for all servers with minute level granularity for the duration between midnight to 4pm GMT on 23 March 2018.

Don't forget to include the cookie file for authentication. Sample response is also shown.

$ curl -X POST -d '{"start": "2018-03-23T15:00:00Z", "end": "2018-03-23T16:00:00Z", "include": [], "outputFormat": "csv", "kind": "servers", "granularity":"minute", "email": "yourEmailAddress"}' https://api.deepcaching.com/reports -b cookie.txt
{ "reportID": "mr-fb34c3d12f9bf475c", "userID": 999, "customerID": 9999, "start": "2018-03-23T15:00:00Z", "end": "2018-03-23T15:30:00Z", "include": [], "kind": "servers", "granularity":"minute", "outputFormat": "csv", "email": "yourEmailAddress", "createdAt": "2018-03-23T15:53:04.365205705Z" }

The input parameters are:

Parameter name Description
start

Start date and time in RFC3339 format. This represents the start time of the usage data interval that you are interested in. Examples:

  • "2018-05-18T08:00:00-07:00" - represents 8am 18 May 2018 in Pacific Time Zone during daylight savings time when the clocks are 7 hours behind GMT.
  • "2018-03-23T15:00:00Z" - represents 3pm 23 March 2018 GMT.
end

End date and time in RFC3339 format. This represents the end time of the usage data interval that you are interested in.

End time must be later than start time.

include

This represents the list of metrics that you are interested in. If empty [], then response will include all metrics.

Valid values are: "buffering", "playing", "savedBw", "rx", "tx".

To retrieve select metrics, specify a list such as ["rx", "tx", "savedBw"]. See the example below for a description of each of these metrics.

outputFormat

Format in which the raw data will be returned.

Valid values are: "csv" or "json".

Use "csv" for CSV (comma separated values) format, and use "json" for JSON format.

kind

This specifies whether you want usage data aggregated for every location, or for each individual ECDN server.

Valid values are: "locations", and "servers".

granularity

This specifies the aggregation unit.

Valid values are "minute" or "hour".

  • When duration of interest specified by "start" and "end" parameters is <= 24 hours, you may retrieve the data aggregated for every "minute" or for every "hour".
  • When duration exceeds 24 hours, the data is aggregated for every hour, and the only valid value is "hour".
email This is the email address where the completion email will be sent after this request is processed. The email will contain the URL to download the file with the usage data.

As shown, the response contains the unique reportID assigned to this request.

Download the report

When processing of request is complete, you will receive an email with a link to file that has the usage data in it. You can click the Download report button to get the file.

This link to the report data file will be valid for 7 days. Beyond this time, you will have to submit a new request to get the usage data.

Example usage data in CSV format

The table below represents the usage data for an ECDN account that has one location Budapest, which has two ECDN server instances named server-001.ecdn-demo.ecdn-ustream.com, and server-002.ecdn-demo.ecdn-ustream.com.

Here's a description of what each metric represents:

timestamp Time of the measurement in UTC.
hostname Hostname of the server.
hostId Host identifier.
locationName Name of the server's location as seen in the ECDN web portal.
locationId Location identifier
playing Number of viewers connected to this server. It represents the concurrency in this time interval ie the number of connections to a specific server, or in case of the locations report it the aggregation of number of connections across all servers configured in that location. Note: ECDN servers cannot identify which employee watched the video stream by connecting to it. It only knows the IP address of machine that connected to it.
buffering Number of viewers experiencing buffering.
rx

This represents the WAN traffic ie amount of data received (video streams downloaded and cached) by the ECDN server(s).

  • In the by-minute granularity report, it represents the average bytes per second received for the given minute internal. Total bytes received in this minute interval = 60 * rx value.
  • In the by-hour granularity report, it represents the sum of the per minute values for each minute within this hour interval. Total bytes received in this hour interval = 60 * rx value.
tx

This represents the downstream LAN traffic ie the amount of data sent out from the ECDN server(s) to all the viewers.

  • In the by-minute granularity report, it represents the average bytes per second sent for the given minute interval. Total bytes sent in this minute interval = 60 * tx value.
  • In the by-hour granularity report, it represents the sum of the per minute values for each minute within this hour interval. Total bytes sent in this hour interval = 60 * tx value.
savedBw

This is a derived metric whose value = (tx - rx) * 60.

It represents the saved bandwidth in bytes over the last time interval.

An aggregate of all these values gives a measure of the amount of WAN bandwidth saved by deploying ECDN servers, for the selected duration.

A positive value, seen during playback, represents that a video chunks were downloaded once over the WAN, and then distributed to all the viewers over the LAN, thereby saving WAN bandwidth.

A negative value usually seen when the ECDN servers are idle, and represents the network traffic due to ECDN servers uploading their metrics and health heartbeat status data.

Notes:

  • Even when there is no playback, there is still network activity in the ECDN server instance such as: heartbeat to ECDN backend, metrics upload, NTP, log forwarding, VPN traffic if remote assistance checkbox is enabled etc.
  • If there is lot of extraneous chatter on the local network, such as when multicast packets are flowing, the "rx" and "tx" values will be skewed, and not accurately represent the ECDN or streaming video traffic.

A report with kind = "servers":

For each server there is one row of data for each minute.

timestamp hostname hostId locationName locationId playing buffering rx tx savedBW = (tx - rx) * 60
2018-03-23T20:31:00Z server-001.ecdn-demo.ecdn-ustream.com 455 Budapest 17 0 0 2419 1602 -49020
2018-03-23T20:31:00Z server-002.ecdn-demo.ecdn-ustream.com 467 Budapest 17 0 0 2694 2339 -21300
2018-03-23T20:32:00Z server-001.ecdn-demo.ecdn-ustream.com 455 Budapest 17 0 0 2282 1530 -45120
2018-03-23T20:32:00Z server-002.ecdn-demo.ecdn-ustream.com 467 Budapest 17 2 0 165189 190873 1541040
2018-03-23T20:33:00Z server-001.ecdn-demo.ecdn-ustream.com 455 Budapest 17 0 0 2496 1624 -52320
2018-03-23T20:33:00Z server-002.ecdn-demo.ecdn-ustream.com 467 Budapest 17 2 0 435119 805249 22207800
2018-03-23T20:34:00Z server-001.ecdn-demo.ecdn-ustream.com 455 Budapest 17 0 0 1694 1277 -25020
2018-03-23T20:34:00Z server-002.ecdn-demo.ecdn-ustream.com 467 Budapest 17 2 0 433732 801739 22080420
2018-03-23T20:35:00Z server-001.ecdn-demo.ecdn-ustream.com 455 Budapest 17 0 0 2349 1549 -48000
2018-03-23T20:35:00Z server-002.ecdn-demo.ecdn-ustream.com 467 Budapest 17 2 0 433963 802286 22099380
2018-03-23T20:36:00Z server-001.ecdn-demo.ecdn-ustream.com 455 Budapest 17 0 0 2442 1607 -50100
2018-03-23T20:36:00Z server-002.ecdn-demo.ecdn-ustream.com 467 Budapest 17 2 0 432888 802216 22159680
2018-03-23T20:37:00Z server-001.ecdn-demo.ecdn-ustream.com 455 Budapest 17 0 0 2277 1516 -45660
2018-03-23T20:37:00Z server-002.ecdn-demo.ecdn-ustream.com 467 Budapest 17 2 0 433578 802535 22137420

A report with kind = "locations":

For each location there is one row of data for each minute.

timestamp locationName locationId playing buffering rx tx savedBW = (tx - rx) * 60
2018-03-23T20:31:00Z Budapest 17 0 0 7284 5278 -120360
2018-03-23T20:32:00Z Budapest 17 2 0 170337 194037 1422000
2018-03-23T20:33:00Z Budapest 17 2 0 440121 808363 22094520
2018-03-23T20:34:00Z Budapest 17 2 0 438270 804624 21981240
2018-03-23T20:35:00Z Budapest 17 2 0 438297 805072 22006500
2018-03-23T20:36:00Z Budapest 17 2 0 438163 805453 22037400
2018-03-23T20:37:00Z Budapest 17 2 0 438417 805576 22029540

Interpreting the data

How many viewers watched a broadcast via the ECDNs?

In the servers report take the maximum value of playing column for each server, and add them up.

In the locations report, take the maximum value of playing column for each location, and add them up.

How many viewers experienced buffering during the broadcast?

Similar to the previous calculation, in the servers report, take the maximum value of the buffering column for each server, and add them up.

In the locations report, take the maximum value of the buffering column for each location, and add them up.

Note: A sustained high value (> 5% of viewership) of buffering usually warrants an investigation. It is not unusual for each site to report some buffering during a broadcast.

On buffering, the IBM Cloud Video player is designed to auto-switch to a lower resolution (bitrate) to provide the best viewing experience. The player can also auto-switch to a higher resolution video, if it detects a higher quality stream is available, and the WAN network has sufficient available bandwidth capacity to deliver this higher bitrate stream.

Viewers from how many locations experienced buffing during the broadcast?

In the locations report, identify all locations with non-zero values in the buffering column. sort | uniq the data to remove duplicates.

Similarly, in the servers report, identify all servers with non-zero values in the buffering column, and then sort | uniq the results. Next further process to eliminate the duplicate locationName values to get the answer.

How much WAN bandwidth was saved by using ECDNs?

Total the savedBw column values in either of the locations or servers report. The sum, in bytes, is the amount of network traffic over the WAN that was avoided by using ECDNs in distributing the video stream data to all the viewers.

Powered by Zendesk