Skip to main content

Method of get late reporting assets

SFlee-2132
Original Poster

Hi Team,

We pull data for our fleet via the API. We call teh various api endpoints on a daily basis and ask for the last 48 hours of data, so every day there is an overlap of 24 hours in case any data is missed.

Even doing that though, often we find that if we go back at month end and pull the whole month we fill in gaps with missing data.

 

What's the best method of pulling the data from the geotab datasets to avoid this?

 

We pull a lot of different calls

events, trips, odometers, engine hours, log status, etc etc. Would really appreciate knowing how we can pull the data using a reported date instead of the event date so we know we've got everything,

 

thanks

 

Join the conversation

You need to be logged in to reply to this post and participate in the Geotab Community discussions.

10 Replies

EishiFUN
Geotabber

Hello @SFlee-2132​ ,

 

Thank you for asking your question in our community. I appreciate your detailed description of your current setup this is a really common pain point and the good news is there is a purpose-built solution for exactly this. I did use an AI tool to put these references together. I am not a developer so if anything doesn't make sense I apologize for that and will be happy to bring in one of our developers to clear anything up.

 

 The Root Cause

 

 Your current approach — querying by date range with a rolling 48-hour overlap— pulls data based on event datetime (when the device recorded the data). The problem is that a device that was offline for hours or days will upload its backlog when it reconnects, and that data arrives at the server after your query window has already moved on. A wider overlap helps but can never fully close this gap.

 

 The Right Approach: GetFeed with Persisted Version Tokens

 

 GetFeed is designed to solve this exactly. Instead of asking "give me everything between date A and date B," it asks "give me everything the server received after this point in the stream." The key distinction: the version token tracks server-receipt order, not event datetime. When a device reconnects and uploads a backlog, that data gets a new version number and will appear in your next GetFeed call — no matter how old the event timestamps are.

 

 How it works:

 

 1. On your first call, pass fromVersion: "0" for a full historical backfill, or omit fromVersion to start from now.

 2. The response includes a toVersion token.

 3. Persist that token to durable storage immediately — before processing the data.

 4. On every subsequent call, pass that toVersion as fromVersion. You receive only records that arrived since your last call.

 5. Repeat indefinitely.

 

 // First call

 { "method": "GetFeed", "params": { "typeName": "Trip", "fromVersion": "0",

 "resultsLimit": 50000 } }

 

 // Response includes toVersion — save it, then next call:

 { "method": "GetFeed", "params": { "typeName": "Trip", "fromVersion":

 "<saved_toVersion>", "resultsLimit": 50000 } }

 

 This covers all your data types: Trip, LogRecord, StatusData, FaultData, ExceptionEvent, DriverChange, and more.

 

 Important Notes on Calculated Data

 

 Trips and ExceptionEvents are calculated records — the server can update or regenerate them as late device data arrives. If a trip is revised (e.g., a driver ID is resolved after the fact), the updated record gets a new version number and will re-appear in your feed with the changes. Your pipeline should be prepared to upsert on record ID rather than insert-only.

 

 One Caveat: fromDate vs fromVersion

 

The docs note that using fromDate is computationally expensive and is only intended as a one-time seed when you first set up the feed. After that, always  use fromVersion. Do not mix the two approaches in steady state.

   

 Polling Cadence

 

 For near-real-time completeness, run your GetFeed calls continuously (or at short intervals). When a response returns a full page of results (resultsLimit  records), call again immediately— there is more data. When a response returns fewer records than the limit, the feed is caught up and you can back off.

 

 Summary

 

 ┌─────────────────────────────┬────────────────────────────────────────────┐

 │   Current approach    │       GetFeed approach       │

 ├─────────────────────────────┼────────────────────────────────────────────┤

 │ Date-range query, event   │ Version-token query, server-receipt order │

 │ datetime          │                      │

 ├─────────────────────────────┼────────────────────────────────────────────┤

 │ Misses late-arriving    │ Catches all data regardless of when device │

 │ backlog data        │ reconnected                │

 ├─────────────────────────────┼────────────────────────────────────────────┤

 │ Requires overlap heuristic │ No overlap needed — the token is the    │

 │               │ bookmark                  │

 ├─────────────────────────────┼────────────────────────────────────────────┤

 │ Gap risk at month-end    │ No gaps as long as token is persisted   │

 └─────────────────────────────┴────────────────────────────────────────────┘

 

The official data feed guide at developers.geotab.com (https://developers.geotab.com/myGeotab/guides/dataFeed/index.html) walks through the full pattern with code examples — worth bookmarking.

   

 Please let us know if you have any other questions. We are here to help.

 

 Have a great day!

 Eishi FUN

 

 Sources:

 - Data Feed Guide — Geotab Developers (https://developers.geotab.com/myGeotab/guides/dataFeed/index.html)

 - GetFeed API Reference — Geotab Developers(https://developers.geotab.com/myGeotab/apiReference/methods/GetFeed/)

 - Result and Rate Limits — Geotab Blog (https://www.geotab.com/blog/result-and-rate-limits/)

SFlee-2132
Original Poster

Thanks for this, this will take me some time to refer to and rebuild. Will post with updates, comments/concerns,

 

Kind Regards,

SFlee-2132
Original Poster

Think I see how it works for events/faults/trips using GetFeed.

 

How does this apply to statusdata and log records?

So if I want to make sure I don't miss odometers/enginehour readings/log records/tyre pressure readings? How can I avoid missing those records that are reported late? But I didn't see the documentation for GetFeed for these data types.

 

Also the VERSION.

The VERSION is database specific? It is call specific as well? ie does FAULT have a differsion VERSION sequence then EVENTS and/or TRIPS?

 

thanks

EishiFUN
Geotabber

Great follow-up questions please feel free to ask any other questions if you have them. My answer ways complied by an AI tool using our official documentation. There is a possibilities there are errors here I didn't identify. So if something doesn't work or doesn't make sense please let me know.

  

 StatusData and LogRecord Are Fully Supported by GetFeed

 

 Yes, GetFeed works for all the data types you mentioned. Here is the complete mapping:

  - GPS/location records → LogRecord

 - Odometer readings → StatusData (filter client-side for DiagnosticOdometerId)

 - Engine hours → StatusData (filter client-side for DiagnosticEngineHoursId)

 - Tyre pressure → StatusData (filter client-side for the relevant diagnostic ID)

 - Fault codes → FaultData

 - Trips → Trip

 - Exception events → ExceptionEvent

 

StatusData is the feed for all diagnostic value readings. The feed returns everything — you then filter on the client side by diagnostic ID for the specific readings you want (odometer, engine hours, tyre pressure, etc.). The Geotab documentation explicitly recommends this approach: do not try to filter at query time, let the feed give you the firehose and filter locally.

 

 One important note on odometer specifically: there are two diagnostic IDs:

 - DiagnosticOdometerId — the ECU-reported raw odometer, corrected by your odometer offset/factor settings

 - DiagnosticOdometerAdjustmentId — a manually adjusted value

  

 For most fleet reporting use cases, DiagnosticOdometerId is what you want.

 

 The Version Token: Per Entity Type, Not Global

  

 This is the key thing to understand: each typeName has its own completely independent version sequence. The toVersion from a StatusData feed call has nothing to do with the toVersion from a FaultData or Trip call. They are separate counters.

   

 In practice this means you need to store and manage a separate token for each data type you are pulling:

 

 LastLogRecordToken   → use with GetFeed typeName: LogRecord

 LastStatusDataToken   → use with GetFeed typeName: StatusData

 LastFaultDataToken   → use with GetFeed typeName: FaultData

 LastTripToken      → use with GetFeed typeName: Trip

 LastExceptionEventToken → use with GetFeed typeName: ExceptionEvent

 

This is confirmed in Geotab's own official .NET sample code (FeedProcessor.cs (https://github.com/Geotab/sdk-dotnet-samples/blob/master/DataFeed/FeedProcessor.cs)), which maintains exactly these separate token variables and passes each one only to its corresponding feed call.

 

 The tokens are also database-specific — a token from one MyGeotab database cannot be used against another.

 

 Putting It Together

  

 Your steady-state pattern for complete data with no gaps:

 

 1. Run GetFeed for each typeName on a polling loop

 2. Store each toVersion in durable storage separately, keyed by typeName

 3. On each call, pass the stored token as fromVersion for that type

 4. If your response is full (hits resultsLimit), call again immediately

 

 Please let us know if you have any other questions. We are here to help.

 

 Have a great day!

 Eishi FUN

 

 Sources:

 - GetFeed API Reference — Geotab Developers (https://developers.geotab.com/myGeotab/apiReference/methods/GetFeed/)

 - Data Feed Guide — Geotab Developers (https://developers.geotab.com/myGeotab/guides/dataFeed/index.html)

 - FeedProcessor.cs sample — Geotab GitHub (https://github.com/Geotab/sdk-dotnet-samples/blob/master/DataFeed/FeedProcessor.cs)

SFlee-2132
Original Poster

Thanks for this. I've got it all working now. But strugglign with a couple of anomolies.

 

I have an asset which between June 16 and June 22 (inclusive) show no engine hours in the status data output. The asset had activity. When I go to the front end and prepare Trips Detail report include the start of trip engine hours and end of trip engine hours the dates between June 16 and June 22 are populated with incrementing engine hours.

 

I'm definitly getting all Status Data, it's all sequentially ordered. I'm not filtering anything on the query either.

SFlee-2132
Original Poster

Additionally, I have assets, which under previous status data calls using dates and filtered for diagnotistic ids I was getting engine hours and odometers. I've changed the call to suit what you suggested and not filter out any of the data on the server side. I'm getting tons of data for these assets - but not odometers now.

I do note that I have all kinds of custom diagnotistic id's which aren't decoded, maybe it's hidden in those? How can I decode them?

SFlee-2132
Original Poster

Is the answer that the status data call, of course, doesn't supply information on the DiagnosticOdometerAdjustmentId nor DiagnosticEngineHoursAdjustmentId?

EishiFUN
Geotabber

Hi @SFlee-2132​ ,

 

Great questions — these anomalies are actually very explainable once you understand a key distinction in how Geotab stores data. I did use an AI tool to help compile these references, so please let me know if anything doesn't match what you're seeing and I'll dig further or loop in one of our developers.

 

Engine hours missing from StatusData June 16–22, but present in Trips Detail

 

These two sources are completely independent, which is why they can disagree.

 

StatusData for DiagnosticEngineHoursId is active data — a raw broadcast received directly from your vehicle's ECU via the CAN bus. If the CAN connection was disrupted during those days, or the ECU simply didn't transmit that parameter, there will be a genuine gap in StatusData with zero records. That is expected and correct behaviour.

 

The Trip.EngineHours field that the Trips Detail report shows is something different entirely. Per the SDK docs, it is "the engine hours as of the end of the trip (measured in seconds)" — a cumulative running total the server maintains and carries forward independently of whether CAN was actively broadcasting during that period. That is why the Trips Detail report can show incrementing values while StatusData has a gap for the same dates.

 

In short: the gap in StatusData is real. The Trips Detail report is not pulling that data from StatusData.

 

Odometers appearing in your old date-range filtered queries but missing in GetFeed

 

I'd suggest checking three things in this order:

1. Diagnostic ID mismatch in your client-side filter. In GetFeed, StatusData records come back with diagnostic: { id: "..." } populated only with the ID — the feed does not hydrate the full Diagnostic object inline. If your filter is comparing against the string "DiagnosticOdometerId" but the actual stored ID for that vehicle is a different internal identifier, the comparison will silently miss every record. Check the raw diagnostic.id values on the records you know represent odometer and make sure your filter matches those exactly.

2. The vehicle may be using a vehicle-specific diagnostic ID, not the standard KnownId. When you used Get<StatusData> with a date range filtered by DiagnosticOdometerId, MyGeotab may have transparently remapped vehicle-specific odometer diagnostics to that standard ID for you. GetFeed returns the actual stored record with the actual stored ID — which for some vehicles is a manufacturer-specific one rather than the standard KnownId. If this is the case, the records are in the feed, just under a different ID.

3. Reporting frequency. Some vehicles only broadcast odometer at low frequency — once per trip, or only on ignition state changes. The records may be in the feed but infrequent enough to look absent if you're only looking at a short window.

 

Decoding your custom/undecoded diagnostic IDs

 

For any diagnostic ID you see in a StatusData record that isn't recognisable, call Get<Diagnostic> with that exact ID. The response will give you:

- Name — human-readable label if one is known (e.g., "Engine Hours", "Odometer")

- Source — where the code originates: SourceGeotabId, SourceJ1939Id, SourceObdId, etc.

- DiagnosticType — SuspectParameter (J1939), ObdFault, PidFault, etc.

- Code — the raw numeric code

- UnitOfMeasure — tells you the unit (metres, seconds, litres, etc.)

 

One practical tip from the official data feed guide: build a local cache of diagnosticId → Diagnostic that you refresh every 24 hours, and use it to label your StatusData records rather than looking up each one on the fly. Many vehicle ECUs broadcast dozens of proprietary codes — if Name comes back as just a numeric string, there is no standard public definition for it and it is a manufacturer-internal code.

 

To directly answer your last question

Yes, both DiagnosticOdometerId and DiagnosticEngineHoursId are real stored StatusData records — they are not synthetic or computed. They only appear when the vehicle ECU actually broadcasts them. The gaps are in the CAN data from the vehicle, not in the API.

 

DiagnosticOdometerId and DiagnosticOdomerAdjustmentId are also two completely separate things — the adjustment ID is a record for a manually configured offset value, distinct from the ECU-sourced reading. Same pattern applies to DiagnosticEngineHoursId vs DiagnosticEngineHoursAdjustmentId.

 

Please let me know if any of this doesn't line up with what you're seeing. Happy to dig further or bring in one of our developers to take a closer look.

 

Have a great day!

Eishi FUN

Sources:

- Data Feed Guide — Geotab Developers (https://developers.geotab.com/myGeotab/guides/dataFeed/index.html)

- GetFeed API Reference — Geotab Developers (https://developers.geotab.com/myGeotab/apiReference/methods/GetFeed/)

- Trip Object Reference — Geotab Developers (https://developers.geotab.com/myGeotab/apiReference/objects/Trip)

- Diagnostic Object Reference — Geotab Developers (https://developers.geotab.com/myGeotab/apiReference/objects/Diagnostic)

SFlee-2132
Original Poster

thanks Eishi,

 

Iv'e peiced this all together now and I can see an improvement. There does come one more question though and it has to do with the adjusted engine hours.

 

I make a call to get an adjusted engine hour reading based on the following trip start dates

 

2026-06-30 21:44:00 3515.98634416667

2026-06-30 04:10:00 4169.41433083333

2026-06-02 22:03:00 2861.14995666667

 

These results were asked for not in the same call, but certainly with in moments of eachother.

Why does the adjusted hour value bounce around? Is this because in the background there is an additional record which suggests an increase in engine hour usage, but then also in the background there is an adjustment record?

 

How could I know when I should, after an adjustment has been made....is there a way to know programmatically that there has been an adjustment and that I should back back a period of days to get renewed values for those records?

 

thanks

EishiFUN
Geotabber

Hi @SFlee-2132​ ,

 

Great follow-up — this one gets into some genuinely nuanced territory. As always, I used an AI tool to help compile this from our internal documentation and SDK references, so please flag anything that doesn't match what you're seeing and I'll bring in one of our developers.

 

Why DiagnosticEngineHoursAdjustmentId bounces across calls

 

The key thing to understand is that DiagnosticEngineHoursAdjustmentId is not a static stored record the way DiagnosticEngineHoursId is. It is the output of the Engine Hours Calculator — a computed value. Specifically:

 

DiagnosticEngineHoursAdjustmentId = last known baseline + accumulated vehicle on-time since that baseline

 

Multiple sources feed into that baseline: the raw ECU value (DiagnosticEngineHoursId), J1939 data, trip data, and any previously applied offsets. If any of those inputs change between your two calls — for example a trip is recalculated in the background, a previously in-progress trip is finalised, or the engineHourOffset on the device is updated — the calculator re-runs and the value returned for the same historical timestamp can be completely different. That is exactly what you're seeing.

 

There is also an important nuance about what happens when someone manually sets engine hours in the MyGeotab UI: the entered value is used only to compute a delta (new value − current known value), and that delta is stored as engineHourOffset on the device. The manually entered value itself is discarded. So what is stored under the hood is an offset, not the absolute value you typed — which is another reason why a past query and a present query for the same timestamp can diverge.

 

So to directly answer your question: yes, your hypothesis is correct. The bouncing is caused by the interplay between the accumulated on-time counter and changes to the underlying baseline — including adjustments.

 

How to detect programmatically that an adjustment has been made

 

There are two reliable signals you can watch via GetFeed:

1. Monitor engineHourOffset on the Device record

The EngineHourOffset field is a property on the device object itself (visible on Go devices via the API). When someone manually adjusts engine hours in the UI, this offset changes and the device entity gets a new version number. Since Device is supported by GetFeed, you can watch for device version changes in your feed. When a device record comes through with a new version, compare the engineHourOffset to your last stored value — if it has changed, you know an adjustment was made and you need to re-fetch historical data for that asset.

2. Monitor the Audit feed

Every modification to a device — including offset changes — creates an entry in the Audit log. GetFeed on Audit will surface these. The Name field on an audit entry indicates the type of operation, and Comment may carry the device reference. This gives you a second signal, and the DateTime on the audit entry tells you approximately when the adjustment was applied.

 

How far back should you go when you detect an adjustment?

 

There is no hard rule here — it depends on how old the trips and EH data affected by the baseline change are. In practice, a conservative approach is to re-fetch a rolling 30-day window for the affected device whenever you detect an offset change. For fleets with seasonal equipment or infrequent operation, you may need to extend that window. The key is to use Get<StatusData> (not GetFeed) to backfill that window for DiagnosticEngineHoursAdjustmentId on that specific device after you detect the change, then upsert the refreshed values into your store.

 

Summary pattern

GetFeed<Device>     → detect engineHourOffset changes

GetFeed<Audit>      → secondary signal, captures who changed what and when

Get<StatusData> (backfill) → re-fetch DiagnosticEngineHoursAdjustmentId for the affected device over a lookback window

 

Please let me know if this makes sense or if you hit anything unexpected — happy to dig further or bring in one of our developers if needed.

 

Have a great day!

Eishi FUN

 

Sources:

- Go9 Device Object Reference — Geotab Developers (https://developers.geotab.com/myGeotab/apiReference/objects/Go9)

- Audit Object Reference — Geotab Developers (https://developers.geotab.com/myGeotab/apiReference/objects/Audit)

- GetFeed API Reference — Geotab Developers (https://developers.geotab.com/myGeotab/apiReference/methods/GetFeed/)

- Data Feed Guide — Geotab Developers (https://developers.geotab.com/myGeotab/guides/dataFeed/index.html)

Still have questions?