ECDN historical usage reports

You can retrieve ECDN usage data for any duration in the past two years. You may submit your requests using REST APIs or via the web portal. All requests are handled asynchronously. Submissions are immediately acknowledged. Processing is queued up. When ready, you will get an email notification with a link to download the raw data file for the specified duration. Two formats are supported: JSON (coming soon) or CSV.

Submit report requests via the web portal

 Click on "REPORTS" in the page header to bring up the "Request report" page.

request report page screenshot

Steps:

  1.  Specify the data range for the duration of interest. You may enter custom values or select a preset from the dropdown. Default is "Today":
    request report page date range dropdown screenshot
  2. Specify the granularity of aggregating the usage data.
    • For date range duration of <= 24 hours, you may pick either Minute or Hour as the aggregation interval.
    • For date range duration > 24 hours, you can only pick Hour as the aggregation interval.
  3. Specify whether you want data to be further aggregated by "Locations" (default), or you want to the data for all ECDN "Servers".
  4. Specify "All metrics" (default) or pick specific ones from the dropdown:
    request report page metrics dropdown screenshot
  5. Specify the format in which data should be returned: "CSV" - comma separated values (default) or "JSON".
  6. Specify the email address to which the report should be sent when ready.
  7. Click the "Request report" button to submit your request.

Your submission will be acknowledged and uniquely identified with a request id.

request report submission acknowledgement screenshot

All report requests are processed asynchronously. When ready, and email will be sent to the specified address, which has a link to download the report. See below for example reports and how to interpret the data.

Submit report requests via REST API

This is a two step process:

  1. Get an authentication token and store it in a cookie.
  2. Submit request using the cookie for authentication.

This article has examples that use the command line tool curl to make the REST API calls. You may also use any other REST tool of your choice.

Authentication

The API uses session based authentication. To get a valid session, send a POST request to /login API giving your email, and password as input parameters. Note, these are the same credentials you use to login to the ECDN web portal.

$ curl -X POST -d 'email=yourEmailAddress&password=yourPassword' -H "Content-Type: application/x-www-form-urlencoded" https://api.deepcaching.com/login -c cookie.txt
{ "success": true }

Replace yourEmailAddress and yourPassword with your email address and password before making the call.

If successful, you will see {"success":true} response, and the API authentication token will be stored in the file cookie.txt in your local directory.

For the ECDN beta environment use the API endpoint: https://beta-api.deepcaching.com/login

Submit request

The report request can be submitted by making a POST request to the /reports API as shown below. This call will retrieve all usage metrics for all servers with minute level granularity for the duration between midnight to 4pm GMT on 16 April 2018.

Don't forget to include the cookie file for authentication. Sample response is also shown.

$ curl -X POST -d '{"start": "2018-04-16T00:00:00Z", "end": "2018-04-16T16:00:00Z", "include": [], "outputFormat": "csv", "kind": "servers", "granularity":"minute", "email": "yourEmailAddress"}' https://api.deepcaching.com/reports -b cookie.txt
{ "reportID": "mr-fb34c3d12f9bf475c", "userID": 999, "customerID": 9999, "start": "2018-04-16T00:00:00Z", "end": "2018-04-16T16:00:00Z", "include": [], "kind": "servers", "granularity":"minute", "outputFormat": "csv", "email": "yourEmailAddress", "createdAt": "2018-04-16T19:53:04.365205705Z" }

The input parameters are:

Parameter name Description
start

Start date and time in RFC3339 format. This represents the start time of the usage data interval that you are interested in. Examples:

  • "2018-05-18T08:00:00-07:00" - represents 8am 18 May 2018 in Pacific Time Zone during daylight savings time when the clocks are 7 hours behind GMT.
  • "2018-04-16T15:00:00Z" - represents 3pm 15 April 2018 GMT.
end

End date and time in RFC3339 format. This represents the end time of the usage data interval that you are interested in.

End time must be later than start time.

include

This represents the list of metrics that you are interested in. If empty [], then response will include all metrics.

Valid values are: "buffering", "playing", "savedBw", "rx", "tx".

To retrieve select metrics, specify a list such as ["rx", "tx", "savedBw"]. See the example below for a description of each of these metrics.

outputFormat

Format in which the raw data will be returned.

Valid values are: "csv" or "json".

Use "csv" for CSV (comma separated values) format, and use "json" for JSON format.

kind

This specifies whether you want usage data aggregated for every location, or for each individual ECDN server.

Valid values are: "locations", and "servers".

granularity

This specifies the aggregation unit.

Valid values are "minute" or "hour".

  • When duration of interest specified by "start" and "end" parameters is <= 24 hours, you may retrieve the data aggregated for every "minute" or for every "hour".
  • When duration exceeds 24 hours, the data is aggregated for every hour, and the only valid value is "hour".
email This is the email address where the completion email will be sent after this request is processed. The email will contain the URL to download the file with the usage data.

As shown, the response contains the unique reportID assigned to this request.

Download the report

When processing of request is complete, you will receive an email with a link to file that has the usage data in it. You can click the Download report button to get the file.

This link to the report data file will be valid for 7 days. Beyond this time, you will have to submit a new request to get the usage data.

Example usage data in CSV format

The table below represents the usage data for an ECDN account that has one location Budapest, which has two ECDN server instances named server-001.ecdn-demo.ecdn-ustream.com, and server-002.ecdn-demo.ecdn-ustream.com.

Here's a description of what each metric represents:

timestamp Time of the measurement in UTC.
hostname Hostname of the server.
hostId Host identifier.
locationName Name of the server's location as seen in the ECDN web portal.
locationId Location identifier
playing Number of viewers connected to this server. It represents the concurrency in this time interval ie the number of connections to a specific server, or in case of the locations report it the aggregation of number of connections across all servers configured in that location. Note: ECDN servers cannot identify which employee watched the video stream by connecting to it. It only knows the IP address of machine that connected to it.
buffering Number of viewers experiencing buffering.
rx

This represents the upstream WAN traffic ie amount of data received (video streams downloaded and cached) by the ECDN server(s).

The value represents the total bytes received in the given time interval (per-minute or per-hour).

rxUnit

Unit of the value in rx column. It is bytes.

tx

This represents the downstream LAN traffic ie the amount of data sent out from the ECDN server(s) to all the viewers.

The value represents the total bytes sent in the given time interval (per-minute or per-hour).

txUnit

Unit of the value in tx column. It is bytes.

savedBw

This is a derived metric whose value = tx - rx.

It represents the saved bandwidth in bytes in the given time interval.

An aggregate of all these values gives a measure of the amount of WAN bandwidth saved by deploying ECDN servers.

A positive value, seen during playback, represents that a video chunks were downloaded once over the WAN, and then distributed to all the viewers over the LAN, thereby saving WAN bandwidth.

A negative value usually seen when the ECDN servers are idle, and represents the network traffic due to ECDN servers uploading their metrics and health heartbeat status data.

Notes:

  • Even when there is no playback, there is still network activity in the ECDN server instance such as: heartbeat to ECDN backend, metrics upload, NTP, log forwarding, VPN traffic if remote assistance checkbox is enabled etc.
  • If there is lot of extraneous chatter on the local network, such as when multicast packets are flowing, the "rx" and "tx" values will be skewed, and not accurately represent the ECDN or streaming video traffic.
savedBwUnit

Unit of the value in savedBw column. It is bytes.

A report with kind = "servers":

For each server there is one row of data for each minute.

timestamp hostname hostId locationName locationId playing buffering rx rxUnit tx txUnit savedBW savedBwUnit
2018-04-16T20:31:00Z server-001.ecdn-demo.ecdn-ustream.com 455 Budapest 17 0 0 2419 bytes 1602 bytes -49020 bytes
2018-04-16T20:31:00Z server-002.ecdn-demo.ecdn-ustream.com 467 Budapest 17 0 0 2694 bytes 2339 bytes -21300 bytes
2018-04-16T20:32:00Z server-001.ecdn-demo.ecdn-ustream.com 455 Budapest 17 0 0 2282 bytes 1530 bytes -45120 bytes
2018-04-16T20:32:00Z server-002.ecdn-demo.ecdn-ustream.com 467 Budapest 17 2 0 165189 bytes 190873 bytes 1541040 bytes
2018-04-16T20:33:00Z server-001.ecdn-demo.ecdn-ustream.com 455 Budapest 17 0 0 2496 bytes 1624 bytes -52320 bytes
2018-04-16T20:33:00Z server-002.ecdn-demo.ecdn-ustream.com 467 Budapest 17 2 0 435119 bytes 805249 bytes 22207800 bytes
2018-04-16T20:34:00Z server-001.ecdn-demo.ecdn-ustream.com 455 Budapest 17 0 0 1694 bytes 1277 bytes -25020 bytes
2018-04-16T20:34:00Z server-002.ecdn-demo.ecdn-ustream.com 467 Budapest 17 2 0 433732 bytes 801739 bytes 22080420 bytes
2018-04-16T20:35:00Z server-001.ecdn-demo.ecdn-ustream.com 455 Budapest 17 0 0 2349 bytes 1549 bytes -48000 bytes
2018-04-16T20:35:00Z server-002.ecdn-demo.ecdn-ustream.com 467 Budapest 17 2 0 433963 bytes 802286 bytes 22099380 bytes
2018-04-16T20:36:00Z server-001.ecdn-demo.ecdn-ustream.com 455 Budapest 17 0 0 2442 bytes 1607 bytes -50100 bytes
2018-04-16T20:36:00Z server-002.ecdn-demo.ecdn-ustream.com 467 Budapest 17 2 0 432888 bytes 802216 bytes 22159680 bytes
2018-04-16T20:37:00Z server-001.ecdn-demo.ecdn-ustream.com 455 Budapest 17 0 0 2277 bytes 1516 bytes -45660 bytes
2018-04-16T20:37:00Z server-002.ecdn-demo.ecdn-ustream.com 467 Budapest 17 2 0 433578 bytes 802535 bytes 22137420 bytes

A report with kind = "locations":

For each location there is one row of data for each minute.

timestamp locationName locationId playing buffering rx rxUnit tx txUnit savedBW savedBwUnit
2018-04-16T20:31:00Z Budapest 17 0 0 7284 bytes 5278 bytes -120360 bytes
2018-04-16T20:32:00Z Budapest 17 2 0 170337 bytes 194037 bytes 1422000 bytes
2018-04-16T20:33:00Z Budapest 17 2 0 440121 bytes 808363 bytes 22094520 bytes
2018-04-16T20:34:00Z Budapest 17 2 0 438270 bytes 804624 bytes 21981240 bytes
2018-04-16T20:35:00Z Budapest 17 2 0 438297 bytes 805072 bytes 22006500 bytes
2018-04-16T20:36:00Z Budapest 17 2 0 438163 bytes 805453 bytes 22037400 bytes
2018-04-16T20:37:00Z Budapest 17 2 0 438417 bytes 805576 bytes 22029540 bytes

Interpreting the data

How many viewers watched a broadcast via the ECDNs?

In the servers report take the maximum value of playing column for each server, and add them up.

In the locations report, take the maximum value of playing column for each location, and add them up.

How many viewers experienced buffering during the broadcast?

Similar to the previous calculation, in the servers report, take the maximum value of the buffering column for each server, and add them up.

In the locations report, take the maximum value of the buffering column for each location, and add them up.

Note: A sustained high value (> 5% of viewership) of buffering usually warrants an investigation. It is not unusual for each site to report some buffering during a broadcast.

On buffering, the IBM Cloud Video player is designed to auto-switch to a lower resolution (bitrate) to provide the best viewing experience. The player can also auto-switch to a higher resolution video, if it detects a higher quality stream is available, and the WAN network has sufficient available bandwidth capacity to deliver this higher bitrate stream.

Viewers from how many locations experienced buffing during the broadcast?

In the locations report, identify all locations with non-zero values in the buffering column. sort | uniq the data to remove duplicates.

Similarly, in the servers report, identify all servers with non-zero values in the buffering column, and then sort | uniq the results. Next further process to eliminate the duplicate locationName values to get the answer.

How much WAN bandwidth was saved by using ECDNs?

Total the savedBw column values in either of the locations or servers report. The sum, in bytes, is the amount of network traffic over the WAN that was avoided by using ECDNs in distributing the video stream data to all the viewers.

Powered by Zendesk