Integration Guide

0 mins to read

API Endpoint Workflows for Records Ingestion and Invalid Records

Introduction

Purpose

This document outlines the non-authentication API records-related endpoint workflows for the Data Intake Gateway (DIG). These include:

  1. Posting records into DIG — /records POST
  2. Querying for invalid records — /invalidrecords GET

The document also includes best practices and sample implementation flow charts for each of the workflows.

NOTE: See Data Intake Gateway (DIG) Guide and the DIG API References for more information on the solution.

Terms and definitions

Term

Definition

Record(s)

Contains context-specific information for a specific device. Usually associated with a specific point in time as a snapshot of data. All records for DIG are considered Custom Telematics Device Records and these terms are used interchangeably.

Custom Telematics Device Record(s)

Data records from custom telematics devices. There are numerous types of records and so the content within each record is contextual to the record type itself.

Accepted Record(s)

A Geotab record that conforms to the record specification (data model) and is accepted in the /record DIG endpoint.

Invalid Records(s)

Records that have correct specifications and data model conformity but are deemed invalid subsequently for a specific reason. These are available for inspection via the /invalidrecords DIG endpoint.

Valid Records(s)

A Geotab record that meets all required criteria to be processed in MyGeotab.

DateTime Timestamp

All DateTime fields for DIG API endpoints must conform to ISO 8601 RFC3339 timestamp format in UTC or offset timezone (e.g. 2020-01-01T01:00:00.000000Z). See Working with Dates section in .

Required reading

Before reading this document, you should read and understand:

  1. The DIG API Authentication Workflow. All workflows identified here require a valid and unexpired bearer token. All DIG API endpoints (that are not authentication related endpoints) require the Authorization: Bearer HTTP header. Thus, integration systems need to implement the authentication workflow alongside the aforementioned flows.
  2. The above Terms and Definitions.
  3. For records there are concepts from the MyGeotab SDK that are relevant:
    1. Unit Of Measure
    2. Working With Dates

Overview

The DIG API provides a primary endpoint to enable integrators to send bulk custom telematics device records to MyGeotab using the /records endpoint.

Additionally, it provides secondary endpoints to help the integrators with troubleshooting, which include:

  1. The /invalidrecords endpoint for invalid records that may have been deemed as invalid or rejected.

Records workflow

Systems that integrate with DIG are expected to send bulk requests containing valid records to the /records API endpoint. You can make these requests as frequently as required for your use case with a consideration of size per request. See the best practices for /records to learn more.

document Image

Figure 1 - Record Endpoint API Workflow

Records endpoint

A call to /records posts a data payload of records using JSON format as an array of record items. All records must contain the following [string] fields:

  1. “DateTime” (timestamp)
  2. “SerialNo” (device serial)
  3. “Type” (record type)

! IMPORTANT: A request containing one or more empty and/or null records will be rejected.

You should provide all other record fields necessary for the given record type. See the DIG OpenAPI YAML spec for a list of supported record types and their respective fields.

NOTE: Some record types may require additional context and understanding. Please see Custom Telematics Device Records in the appendix below or reach out to integrations@geotab.com for support.

NOTE: Data accepted by DIG is not immediately available in MyGeotab. Backend solution processing takes place, and the data becomes available in the target database(s) as quickly as possible. See the Data Processing Latency section in the Appendix for additional detail and considerations.

Term

Definition

Accepted Request

A valid request to /records that conforms to the HTTP POST specification requirements, contains properly formatted JSON, and contains acceptable records. It returns an HTTP 202 ACCEPTED response.

Invalid Request

A request to /records that is rejected because it is not valid. The entire request is rejected with HTTP 400 BAD_REQUEST response.

See a sample workflow implementation flowchart below:

document Image

Figure 1 - Record Endpoint Sample Implementation Workflow

Endpoint

[https://dig.geotab.com:443/records] POST

Request Body

Array of valid records items

Returns

If the request is accepted:

  1. 202 ACCEPTED with
    1. Data: returns a request ID
    2. Error: <empty>

If the request is NOT accepted:

  1. 400 BAD_REQUEST or
  2. 413 PAYLOAD_TOO_LARGE or
  3. 401 UNAUTHORIZED
  4. with

    1. Data: <empty>
    2. Error: populated with error message(s)
    3. The above class of responses should not be automatically retried, as they require corrective action.

      Other HTTP response codes should be retried per their specifications:

  5. 4XX or 5XX; see “API References” and “HTTP Response Codes” sections in
  6. Data integrity considerations

    Please consider the following:

    NOTE: The solution ignores extraneous data fields in records that do not conform to the record specification. Additionally, as per the OpenAPI format specification, field names are case-sensitive and data MAY be ignored or dropped if it does not conform to the field name casing specified in the OpenAPI YAML specification. Geotab recommends all JSON payload field names use PascalCase naming.

    NOTE: Optional fields with null values or that are missing entirely are set to a default value, following the Default values of C# types. Depending on your device data reporting workflows and the record type, it may be required for an integration to explicitly include fields and set their values to avoid misinterpretation.

    Size and frequency of bulk requests

    The frequency and size of requests can have competing priorities. Consider that typically for any ingestion system, smaller bulk requests will have lower throughput performance, and larger bulk requests will have higher throughput. This is a matter of maximizing payload efficiency per request (TCP & HTTP request overhead) and minimizing latency.

    We recommend sending requests with between 200 - 500 records on average for optimal performance in steady state. Typically, the maximum number of records threshold for your integration could be up to 1000 records per request. Larger requests are also acceptable should your system experience a backlog or burst and needs to recover.

    There is no "one size fits all". Integrators need to determine what works best for their systems; they will typically employ logic for both “number of records per request” and “time since last request”. Practically, this means bulking requests up to a certain maximum size, or up until a maximum duration since the last request. The example flowchart above uses sane default starting values.

    NOTE: If you have experienced a technical issue and need to send larger than normal bulk processing requests to recover (beyond your current active production workload rates), we request that you schedule this during off-peak hours (relative to the timezone of the region). Please contact Geotab Support before sending extremely large requests for an extended period of time that are out of the ordinary. Typically, a minimum of two business days advance notice is necessary.

    Curl examples

    Generic Request

    curl --request POST \

    --url https://dig.geotab.com:443/records \

    --header 'Authorization: Bearer eyJh--Bearer-token-clipped-for-brevity-TCsIA' \

    --header 'content-type: application/json' \

    --data '[{RECORD_1}, {RECORD_2}, {...}, ...]'

    Success Request & Response

    1. RECORD_X is a GenericStatusRecord
    2. Successful POST example returns 202 ACCEPTED
    3. {"DateTime":"2020-01-01T01:00:00Z","SerialNo":"xxxxxxx","Type":"GenericStatusRecord","Code":X,"Value":X}

      Response:

      {

      "Error": [],

      "Data": "6184f7b2-clipped-for-brevity-fa93c1e53a5d"

      }

      Rejected Requests & Responses

      Invalid DateTime:

    4. RECORD_X is a VinRecord with an unspecified time zone
    5. Non-successful POST is rejected as 400 BAD_REQUEST

    {"DateTime":"2020-01-01T01:00:00","SerialNo":"xxxxxxx","Type":"VinRecord","VehicleIdentificationNumber":X}

    Response:

    {

    "Error": [

    "Bad Request: one or more validation errors occurred",

    "Encountered one or more records having their DateTime field with an unspecified time zone"

    ],

    "Data": null

    }

    Invalid JSON:

    1. RECORD_X is a BaseRecord with invalid JSON (wrong quotes)
    2. Non-successful POST is rejected as 400 BAD_REQUEST

    {"DateTime":’2022-10-13T16:02:22.2216677Z’,"SerialNo":’xxxxxxx’,"Type":’None’}

    Response:

    {

    "Error": [

    "Bad Request: one or more validation errors occurred",

    "Encountered an error in the JSON payload, verify it adheres to RFC8259"

    ],

    "Data": null

    }

    NOTE: For errors related to an invalid date, see the “Working with Dates” section in the Data Intake Gateway Guide.


    Invalid records workflow

    DIG provides the /invalidrecords endpoint to query for invalid records that may have been marked as invalid or rejected for various reasons. Systems that integrate with DIG should periodically query this endpoint to inspect, identify, and correct any problems. Records that are corrected can then be resent to the DIG /records endpoint.

    document Image

    Figure 3 - Invalid Record API Endpoint Workflow

    Invalid records accumulate within DIG as they are processed and identified through the /records workflow. Invalid records are not sent to MyGeotab; they become available for your account to query and inspect.

    NOTE: Data records accepted by DIG that are marked as invalid will be available at this endpoint as quickly as possible. See the Data Processing Latency section in the Appendix for additional detail and considerations.

    A request to /invalidrecords gets a data payload that may contain invalid records if they exist. The workflow and usage of the endpoint supports some basic parameters for controlling the maximum number of results per response and the next result key for walking results (aka pagination) and incremental calls over time. The response also includes metadata that is useful for the integrator to implement this workflow with the system.

    The array of returned invalid records is a consistent structure that includes their type, original field values, the reason why they are considered invalid, along with other metadata.

    DIG will expire invalid records after a period of time. To avoid data loss, it is expected that your integration queries this endpoint periodically to save them for manual inspection.

    To recap: The integrator must periodically query /invalidrecords to investigate erroneous data, correct problems, and resend the data.

    document Image

    Figure 4 - Invalid Records Endpoint Sample Implementation Workflow

    Invalid records endpoint

    Endpoint

    [https://dig.geotab.com:443/invalidrecords] GET

    Params

    NextResultKey: set to 0 initially; subsequent requests should set this to the previous request’s NextResultKey response value;.

    NumberOfResults: the maximum number of invalid records to return for this request

    Returns

    If the request is accepted:

    1. 200 OK with
      1. Data (for successful responses):
        1. NextResultKey: use this for the next request’s NextResultKey parameter.
        2. CurrentResultKey: the value of NextResultKey that was specified in the current request.
        3. TotalNumberOfInvalidRecords: the total number of unexpired invalid records for your account.
        4. CurrentNumberOfInvalidRecords: the number of invalid records returned in this response.
        5. InvalidRecords: [array] of invalid records (if any) with metadata.
      2. Error (for unsuccessful responses, populated with error message)

    If the request is NOT accepted:

    1. 400 BAD_REQUEST or
    2. 401 UNAUTHORIZED

    with

  7. Data: <empty>
  8. Error: populated with error message(s)
  9. The above class of responses should not be automatically retried, as they require corrective action.

    Other HTTP response codes should be retried per their specifications:

  • 4XX or 5XX; see “API References” and “HTTP Response Codes” sections in
  • Reasons for records to be marked as invalid and expected actions

    Geotab recommends that all invalid records provided by DIG are stored for manual inspection and case-by-case assessment before resending or discarding. There are many reasons why a record may be marked as invalid. The system assesses validity in multiple stages. The most common reasons include, but are not limited to:

    1. Invalid Serial Number Sending a record for a device that is invalid, unprovisioned, not active, or not owned by your account.
    2. Missing Mandatory Data Field Some record fields are mandatory. (contextual to the record type itself; does not apply to SerialNo, DateTime, and Type fields)
    3. Bad or Corrupt Data Some record types have data value validations and restrictions per industry specification, DIG API specification, or some other requirement; additionally, some data types require a specific encoding.

    All scenarios must have some action performed and a decision by the integrator whether to retry the data. Before retrying, actions may include:

    1. Finishing administrative tasks (billing, device provisioning)
    2. Correcting corrupt data from devices at the source
    3. Fixing issues in the integration implementation’s data processing
    4. Etc.

    A final decision to discard the data may also be the outcome, and will depend on your business requirements.

    Explanation for NextResultKey and CurrentResultKey

    There is a relationship between:

    1. NextResultKey in a request and CurrentResultKey in the response.
    2. NextResultKey from a response and NextResultKey in the subsequent request.

    NextResultKey can be thought of as a pagination key that will unpredictably increase monotonically, and must be tracked and maintained by the integration, as it is chained from request to request. If this tracker is lost, the integration will need to start over with 0 and would be forced to reprocess duplicate invalid records. Note that this value is not guaranteed to always increase.

    CurrentResultKey in a response is simply metadata equal to the value of NextResultKey from the associated request.

    NOTE: The NextResultKey value in the response is not predictable. Your integration implementation must take care to adhere precisely to this value in subsequent requests. The value in a response may be the same as CurrentResultKey or some numerical value higher than the current result key. The primary function of this value is for the subsequent request’s NextResultKey parameter.

    The following simplified sequence diagram can help you understand these relationships:

    document Image

    Figure 5 - Invalid Records ResultKey Explanation

    Explanation for CurrentNumberOfInvalidRecords and TotalNumberOfInvalidRecords

    Both of these response fields are simple counters for the number of invalid records respectively.

    CurrentNumberOfInvalidRecords will always be a subset of the TotalNumberOfInvalidRecords, and will be less than or equal to the request parameter NumberOfResults. While CurrentNumberOfInvalidRecords is >=0, there may be more invalid records returned in subsequent requests to /invalidrecords.

    TotalNumberOfInvalidRecords will increase or decrease over time, but always reflects DIG’s state of total invalid records for your account at the time the request was made. It increases when there are new invalid records processed and it decreases when invalid records expire.

    NOTE: When the CurrentNumberOfInvalidRecords response value equals zero, we can assert that there are no more invalid records at this time. The integration can temporarily hold querying for more invalid records until the next query cycle time.

    NOTE: TotalNumberOfInvalidRecords may suddenly increase or decrease in response to new records entering the system and/or expired records exiting the system. Implementations using an iteration algorithm to walk invalid records over multiple serial requests need to take this case into account.

    Curl example

    Request

    curl --request GET \

    --url 'https://dig.geotab.com:443/invalidrecords?NextResultKey=0&NumberOfResults=100' \

    --header 'Authorization: Bearer eyJh--Bearer-token-clipped-for-brevity-TCsIA'

    Response

    {

    "Error": [],

    "Data": {

    "NextResultKey": 10514,

    "CurrentResultKey": 0,

    "TotalNumberOfInvalidRecords": 200,

    "CurrentNumberOfInvalidRecords": 100,

    "InvalidRecords": [{INVALID_RECORD_1}, {INVALID_RECORD_2}, ..., {INVALID_RECORD_100}]

    }

    }

    Where: INVALID_RECORD_X (in example) is an invalid record denoting metadata as well as the original record content ingressed to DIG /records endpoint. The metadata will include a description for why this record was not valid:

    {"TimeStamp":"2020-01-01T01:00:00Z","Cause":"some reason","UserId":"<>", “BaseRecord”: {...}}

    Example of a single invalid record:

    {

    "Error": [],

    "Data": {

    ...,

    "InvalidRecords": [

    {

    "BaseRecord": {

    "Code": 255,

    "Value": 0,

    "DateTime": "2021-01-01T05:00:00Z",

    "SerialNo": "dummy",

    "Type": "GenericStatusRecord"

    },

    "Cause": "The record was not accepted because the serial number is unknown or has a formatting issue. This may be an unprovisioned device or we are experiencing a delay to provision.",

    "TimeStamp": "2021-07-16T15:00:14.3370137Z",

    "UserId": "<snip>"

    }

    ]

    }

    }

    Invalid record expiration & usage expectations

    The set of invalid records for your account becomes unavailable once they expire. Typically, invalid records should be available for approximately 48 hours from the time they were identified and processed as invalid records. This expiration is tracked per invalid record.

    NOTE: To avoid loss of data, we recommend that you query for invalid records every hour, 24/7. Geotab does not provide a hard time guarantee, and invalid records may expire within the system sooner or later than expected.

    ! IMPORTANT: The invalid records workflow is intended as a feedback loop to allow integration systems to observe errors in record data or device provisioning and make corrections to avoid data loss. The volume of invalid record data is expected to be minimal. You must take ongoing care to keep your invalid records minimized. If the system detects an unexpected increase in invalid records, Geotab Support may notify you and reserves the right to purge this data if required. If the ingestion surge of invalid records is not quickly mitigated, Geotab also reserves the right to temporarily deactivate your DIG account.

    Workflow walkthrough

    The following section displays a more in-depth example of several /invalidrecord requests, demonstrating what we have learned.

    At a high level, we will work through the following:

    1. As an initial state, no invalid records.
    2. Then there are 8 invalid records; query for 5 (get 5).
    3. No new invalid records, but still more to get; query for 5 (get 3).
    4. We have queried for all the invalid records and there are no new ones; query again for 5 (get 0).
    5. Workflow takes a break querying for 1 hour.
    6. Resume querying, still no new invalid records.
    7. Workflow continues querying for several days, still no new invalid records.
    8. Invalid records expire.

    1

    Initial State: No Invalid Records.

    Request

    curl --request GET \

    --url 'https://dig.geotab.com:443/invalidrecords?NextResultKey=0&NumberOfResults=5' \

    --header 'Authorization: Bearer eyJh--Bearer-token-clipped-TCsIA'

    Response

    {

    "Error": [],

    "Data": {

    "NextResultKey": 0,

    "CurrentResultKey": 0,

    "TotalNumberOfInvalidRecords": 0,

    "CurrentNumberOfInvalidRecords": 0,

    "InvalidRecords": []

    }

    }

    Comments:

    1. No invalid records were returned, so CurrentNumberOfInvalidRecords=0
    2. No invalid records exist yet for this account, so TotalNumberOfInvalidRecords=0
    3. Notice an unpredictable NextResultKey was returned, to be used in the next request

    2

    There are 8 new Invalid Records from the /records pipeline of your integration.

    Request

    curl --request GET \

    --url 'https://dig.geotab.com:443/invalidrecords?NextResultKey=0&NumberOfResults=5' \

    --header 'Authorization: Bearer eyJh--Bearer-token-clipped-TCsIA'

    Response

    {

    "Error": [],

    "Data": {

    "NextResultKey": 51886,

    "CurrentResultKey": 0,

    "TotalNumberOfInvalidRecords": 8,

    "CurrentNumberOfInvalidRecords": 5,

    "InvalidRecords": [

    {

    "BaseRecord": {

    "Code": 255,

    "Value": 0,

    "DateTime": "2021-01-01T05:00:00Z",

    "SerialNo": "dummy",

    "Type": "GenericStatusRecord"

    },

    "Cause": "The record was not accepted because the serial number is unknown or has a formatting issue. This may be an unprovisioned device or we are experiencing a delay to provision.",

    "TimeStamp": "2021-07-16T15:00:14.3370137Z",

    "UserId": "<snip>"

    },

    {...},

    {...},

    {...},

    {...}

    ]

    }

    }

    Comments:

  • 5 invalid records were returned, so CurrentNumberOfInvalidRecords=5, and the “InvalidRecords” array is populated with the record that was invalid as well as some metadata
  • 8 invalid records exist for this account, so TotalNumberOfInvalidRecords=8
  • Notice again an unpredictable NextResultKey was returned, to be used in the next request
  • 3

    There are no new Invalid Records from the /records pipeline of your integration. However, some still remain to be queried.

    Request

    curl --request GET \

    --url 'https://dig.geotab.com:443/invalidrecords?NextResultKey=51886&NumberOfResults=5' \

    --header 'Authorization: Bearer eyJh--Bearer-token-clipped-TCsIA'

    Response

    {

    "Error": [],

    "Data": {

    "NextResultKey": 51889,

    "CurrentResultKey": 51886,

    "TotalNumberOfInvalidRecords": 8,

    "CurrentNumberOfInvalidRecords": 3,

    "InvalidRecords": [

    {...},

    {...},

    {...}

    ]

    }

    }

    Comments:

  • 3 invalid records were returned, so CurrentNumberOfInvalidRecords=3, and the “InvalidRecords” array is populated
  • Still 8 invalid records exist for this account, so TotalNumberOfInvalidRecords=8
  • Notice again an unpredictable NextResultKey was returned, to be used in the next request
  • 4

    There are no new Invalid Records from the /records pipeline of your integration. However, we query again (and again) until no invalid records are returned.

    Request

    curl --request GET \

    --url 'https://dig.geotab.com:443/invalidrecords?NextResultKey=51889&NumberOfResults=5' \

    --header 'Authorization: Bearer eyJh--Bearer-token-clipped-TCsIA'

    Response

    {

    "Error": [],

    "Data": {

    "NextResultKey": 51889,

    "CurrentResultKey": 51889,

    "TotalNumberOfInvalidRecords": 8,

    "CurrentNumberOfInvalidRecords": 0,

    "InvalidRecords": []

    }

    }

    Comments:

  • No invalid records were returned, so CurrentNumberOfInvalidRecords=0
  • Still 8 invalid records exist for this account, so TotalNumberOfInvalidRecords=8
  • 5

    At this point, the integration pauses (waits) for 1 hour before checking again.

    6

    There are still no new Invalid Records from the /records pipeline of your integration, but it is time to check again.

    Request

    curl --request GET \

    --url 'https://dig.geotab.com:443/invalidrecords?NextResultKey=51889&NumberOfResults=5' \

    --header 'Authorization: Bearer eyJh--Bearer-token-clipped-TCsIA'

    Response

    {

    "Error": [],

    "Data": {

    "NextResultKey": 51889,

    "CurrentResultKey": 51889,

    "TotalNumberOfInvalidRecords": 8,

    "CurrentNumberOfInvalidRecords": 0,

    "InvalidRecords": []

    }

    }

    Comments:

  • No invalid records were returned, so CurrentNumberOfInvalidRecords=0
  • Still 8 invalid records exist for this account, so TotalNumberOfInvalidRecords=8
  • 7

    The integration continues to query every hour.

    8

    Imagine there were no new invalid records for several days, and your invalid records have expired in DIG. Now there are no invalid records.

    Request

    curl --request GET \

    --url 'https://dig.geotab.com:443/invalidrecords?NextResultKey=51889&NumberOfResults=5' \

    --header 'Authorization: Bearer eyJh--Bearer-token-clipped-TCsIA'

    Response

    {

    "Error": [],

    "Data": {

    "NextResultKey": 51889,

    "CurrentResultKey": 51889,

    "TotalNumberOfInvalidRecords": 0,

    "CurrentNumberOfInvalidRecords": 0,

    "InvalidRecords": []

    }

    }

    Comments:

  • Back to no invalid records existing for this account, so TotalNumberOfInvalidRecords=0
  • Best practices & usage expectations

    /authentication/*

    Review how your integrations will implement the authentication workflows by understanding the API Authentication Workflow Service Account Best Practices as well as the API Authentication and Token Management Implementation Strategies.

    Minimizing redundant & duplicate data into /records

    Some devices may output “noisy” (repetitive) data even when there is no change. To scale your integration and achieve optimal performance with the Geotab backend, we recommend minimizing duplicate records from your custom telematics devices.

    Records entering the Data Intake Gateway should be for new or changed data from previous record states, with some exceptions such as infrequent status record updates that may necessitate periodic (low frequency) updates.

    Having a minimal and lean set of data (and still sufficiently granular for your business needs) is optimal.

    Separation of the /records Pipeline & /invalidrecords Processes

    Geotab recommends for simplicity that integrators consider the primary inline pipeline path of sending bulk records to DIG to be a separate process from the secondary system that queries periodically for invalid records.

    This allows /records requests to “fire and forget” with high performance and scale using Fast ACK, when the response is 202 ACCEPTED. No further auditing or tracking from this portion of the integration would necessarily be required.

    A separate secondary process can query /invalidrecords periodically to re-ingest records that were marked as invalid. These records should be saved to storage for human assessment and correction before resending those records for re-processing.

    Rate limits

    The core DIG API workflow endpoints outlined in this document are authenticated endpoints (protected by JWT bearer token from the Authentication and Token Management workflows), so there are no rate limits for the /records and /invalidrecords endpoints at this time.

    Limitations

    This section describes any limitations related to this service.

    It is currently not possible to insert (import) historical data older than 365 days.

    Appendix

    Data processing latency for valid and invalid records

    If a request to the /records endpoint is accepted (202), your high-performance integration can assume the data was accepted. These records will be processed internally and delivered to the appropriate MyGeotab database(s) as quickly as possible, or be subsequently marked as invalid.

    The determination for whether a record is valid or invalid is made as quickly as possible. Due to multiple variables within the system and multiple reasons for records to be marked as invalid, Geotab cannot provide a hard time guarantee.

    The /invalidrecords endpoint can be periodically queried to determine if any records were marked as invalid and need attention. Invalid records will be detected, processed, and made available to this endpoint as quickly as possible.

    Customers can rest assured that data is processed on high-performance infrastructure and becomes available promptly with minimal latency.

    Custom telematics device records

    This section outlines any supporting information for custom telematics device records that can be helpful beyond what the OpenAPI yaml specification supplies.

    Please adhere to this specification to align with record fields and their object models. To make requests against DIG, this document provides the Records Workflow section above describing how to generally construct API requests to send records. The sections and samples below do not duplicate the record model specification nor iterate the matrix of possible sample requests.

    The records workflow and the API specification should provide sufficient documentation to implement a custom telematics device integration with Geotab.

    The sections below are additional documentation that goes beyond the specification and DIG API invocation. Where needed for specific record types, we will highlight scenarios and correlate with the MyGeotab platform.

    Acceleration record

    A record that contains accelerometer data. Axis acceleration units are in milli-g (1000mG = 1G of gravitational force). For more information about g-force, refer to our What is g-force blog article.

    Axis

    Description

    X-axis

    The X-axis acceleration (forward or backward) in milli-g.

    Y-axis

    The Y-axis acceleration (right or left) in milli-g.

    Z-axis

    The Z-axis acceleration (up or down) in milli-g.

    Sample Acceleration Record

    {

    "DateTime": "2020-02-19T15:50:47.0923205Z",

    "SerialNo": "xxxxxxx",

    "Type": "AccelerationRecord",

    "X": 3,

    "Y": -5,

    "Z": 0

    }

    BinaryRecord: Storage of arbitrary binary/string data

    Consider the following sample of arbitrary data that an integrator would like to store in the Geotab ecosystem that falls outside the supported MyGeotab record data schemas. This can be unique per custom telematics device for a particular time and date.

    This flexible binary record is useful as well for prototyping solutions integration systems to verify how data can leave and be retrieved.

    Data sample

    Main Battery=12.89;Backup Battery=0;Box Serial Number=######;Firmware Version=0001_tr00.SE7.G30.NN(Aug 1 2020 14:12:47)#Dis WxC:xC000002 GPS:S5-P4.0;Waypoints Usage=;SIM=XXXXXXX;Cell=;OTAB=N;

    Data Base64 encoded

    TWFpbiBCYXR0ZXJ5PTEyLjg5O0JhY2t1cCBCYXR0ZXJ5PTA7Qm94IFNlcmlhbCBOdW1iZXI9IyMjIyMjO0Zpcm13YXJlIFZlcnNpb249MDAwMV90cjAwLlNFNy5HMzAuTk4oQXVnICAxIDIwMjAgMTQ6MTI6NDcpI0RpcyBXeEM6eEMwMDAwMDIgR1BTOlM1LVA0LjA7V2F5cG9pbnRzIFVzYWdlPTtTSU09WFhYWFhYWDtDZWxsPTtPVEFCPU47Cg==

    DIG API request

    This data can be sent using the BinaryRecord record type; set the “Data” field to the arbitrary data, Base64 encoded.

    Retrieving the binary data

    This data can be retrieved from MyGeotab using the MyGeotab SDK using GetFeed as shown below:

    MyGeotab API Request

    api.call("GetFeed", {

    "typeName": "BinaryData",

    "resultsLimit": 10,

    "fromVersion": "0000000000000000",

    "search": {

    "deviceSearch": {

    "id": "XXXXX"

    }

    }

    }, function(result) {

    console.log("Done: ", result);

    }, function(e) {

    console.error("Failed:", e);

    });

    MyGeotab API Response

    Object { ... }

    -data: Array[1]

    -0: Object

    binaryType: "ThirdPartyData"

    controller: "ControllerNoneId"

    data: "TWFpbiBCYXR0ZXJ5PTEyLj(snipped)WxsPTtPVEFCPU47Cg=="

    dateTime: "2020-10-27T19:00:00.000Z"

    -device: Object

    id: "XXXXX"

    id: "a7A7zKgoMRrfx2Mv0EeVdAA"

    version: "0000000000100de7"

    length: 1

    toVersion: "0000000000100de7"

    DriverChangeRecord: Driver Assignment

    This functionality is equivalent to the Driver Assignment feature in MyGeotab and similar to the blogged NFC Driver ID scenario.

    For custom telematics device integrations, we can consider the goal of sending a “Driver Assignment” event from a key swipe or other custom hardware or driver operation.

    NOTE: To use this feature, it is expected that the Driver ID key value is configured within MyGeotab. See the How to Assign a Driver ID Key to a Driver in MyGeotab section. The Driver ID key value string entered into MyGeotab must match the driver key fob serial number value prior to being encoded as base64, as it is processed by the custom hardware solution and integration.

    DIG API Request

    Integrators can use the DriverChangeRecord for driver assignment.

    Supported Key Types

    Key Type Value

    Key Type Label MyGeotab Mapping

    Driver ID

    84

    Custom

    Unique string, Base64 encoded

    MyGeotab UI Outcomes

    The following table outlines outcomes before with no driver assigned.

    document Image

    The following table outlines outcomes after with a new driver assigned.


    Fault records

    There are different defined protocols by the Society of Automotive Engineers SAE group for fault codes depending on the age and type of asset. The following record types enable the ingestion of fault data records into the Geotab platform through DIG.

    Geotab Fault Record Name

    Standards and Fault Codes

    Supplementary Notes

    OBD II Fault Record

    OBD II Codes Defined

    OBD II Trouble Codes

    OBD 2 Explained

    J1939 Fault Record

    SAE

    J1708 Fault Record

    SAE

    J1708 is the physical specification, and J1587 is the communication protocol. They are commonly both referred to as J1708 in the industry.

    Generic Fault Record

    N/A – Geotab’s generic fault record for proprietary implementations.

    There are different input criteria based on the protocol utilized by the connected vehicle, so integrators should utilize as appropriate. Faults captured through these record types are visible by navigating to Engine Faults on MyGeotab’s menu.

    Each record model and its properties are denoted in the DIG API on Github: https://github.com/Geotab/data-intake-gateway.

    The following outlines the context and specific explanations:

    1. Default State Values:
      1. Fault and Lamp States default to false.
    2. Fault State:
      1. Translation is mapped to these fault states in the MyGeotab SDK
      2. Currently, MyGeotab supports if a fault state is active or not active.
    3. Lamp State:
      1. Translation is mapped to these lamp states in the MyGeotab SDK
    4. Failure Mode Identifier:
      1. Translation is mapped to the “code” (integer) in the MyGeotab SDK

    See MyGeotab CTD Diagnostics – Status and Fault Codes explained for information on obtaining the correct code value.

    GpsRecord

    Representing your custom device on the live Map and rules-engine processing

    Most integrations desire the custom telematics device to indicate its current state to MyGeotab users (via live Map) or the MyGeotab application rules engine (to process exceptions).

    The custom telematics device accomplishes that by manipulating the isIgnitionOn and isGpsValid properties at the same time. Below is a sample GpsRecord:

    {

    "DateTime": "2020-01-01T01:00:00Z",

    "SerialNo": "xxxxxxx",

    "Type": "GpsRecord",

    "IsGpsValid": True,

    "IsIgnitionOn": True,

    "Latitude": 43.641907,

    "Longitude": -79.455228,

    "Speed": 40

    }

    To indicate that an asset has stopped moving (either at the end of a trip or the leg of a journey), the custom telematics device needs to send records in the following sequence, with its properties set as follows:

    1. A record with isIgnitionOn=false and isGpsValid=true.
    2. A record with (a) identical GPS coordinates as previous with (b) a timestamp difference >200 seconds.
    3. A record with isIgnitionOn=false and isGpsValid=false to force the live Map to interpret a stop.

    The custom telematics device will indicate that it is moving by sending a record with isIgnitionOn=true and isGpsValid=true.

    NOTE: When isGpsValid is set to false, the rules engine considers the GPS coordinates for this record as inaccurate and ignores them. By default, a GO device deems a coordinate as accurate if there is a GPS latch with at least five satellites, at which point isGpsValid is set to true. If there is less than that, the coordinates are still sent but with isGpsValid=false, which means the rules engine will possibly ignore them.

    On the Geotab live Map, the Iconography for vehicles states are documented in the Map Icons section.

    StatusRecord and GenericStatusRecord

    Both StatusRecord and GenericStatusRecord allow status data to be sent to MyGeotab. The difference between the two is that StatusRecords map to legacy third-party source diagnostics, whereas GenericStatusRecords map to telematics device source diagnostics.

    1. To find the diagnostic code for GenericStatusRecords, refer to the spreadsheet.
    2. To find the diagnostic code for StatusRecords, select Diagnostics from the Engine and Device tab in MyGeotab. Then, choose Third-party for the source filter.

    In order for an integration to leverage status records, we recommend reviewing the MyGeotab diagnostics available, comparing and mapping to your system, experimenting/testing, and validating the expected behavior from the source system and ultimately how this is exposed within MyGeotab for your requirements.

    NOTE: Geotab recommends using GenericStatusRecords, as third-party source diagnostics via the legacy StatusRecords will be deprecated in the future.

    See MyGeotab CTD Diagnostics – Status and Fault Codes explained for information on obtaining the correct code value.

    MyGeotab CTD Diagnostics – Status and Fault Codes explained

    Status data and faults are both considered diagnostics within Geotab systems, and all diagnostics have a unique code value that identifies them. Refer to How to find the Telematics Diagnostic name from the codes | Geotab Knowledge Article for an overview of how the entire list of MyGeotab diagnostics can be exported in a report.

    ✱ NOTE: Use source Telematics Device.

    Use the search bar to filter for diagnostics by their name. Alternatively, lists these items.

    As it pertains to custom telematics devices, the code value is applicable for the following.

    Record Type

    Diagnostic Code Value Range

    Supplementary Notes

    GenericFaultRecord

    Within: 128 to 1999

    Inclusive

    GenericStatusRecord

    Outside of: 128 to 1999

    Exclusive

    scroll-up