blf: interpret object timestamps as signed 64-bit values - #2101
Open
FlorianMickler wants to merge 1 commit into
Open
blf: interpret object timestamps as signed 64-bit values#2101FlorianMickler wants to merge 1 commit into
FlorianMickler wants to merge 1 commit into
Conversation
Change BLF object header timestamp fields from uint64 (Q) to int64 (q) for both V1 and V2 object headers. Observed CANape-generated BLF files containing messages before the recording start timestamp. When interpreted as unsigned values, these timestamps wrap around and appear approximately 2^64 ns in the future, resulting in invalid timestamps (e.g. year 2611) and NaT values in pandas. Using signed 64-bit timestamps correctly preserves negative recording-relative offsets while still providing a usable range of approximately ±292 years at nanosecond resolution. Root cause: A timestamp of -60 s was decoded as 2^64 ns - 60 s which produced timestamps approximately 584 years in the future. The resulting values exceeded the valid pandas datetime range and appeared as NaT.
Contributor
|
Tick the box to add this pull request to the merge queue (same as
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change BLF object header timestamp fields from uint64 (Q) to int64 (q) for both V1 and V2 object headers.
Observed CANape-generated BLF files containing messages before the recording start timestamp. When interpreted as unsigned values, these timestamps wrap around and appear approximately 2^64 ns in the future, resulting in invalid timestamps (e.g. year 2611) and NaT values in pandas.
Using signed 64-bit timestamps correctly preserves negative recording-relative offsets while still providing a usable range of approximately ±292 years at nanosecond resolution.
Root cause:
A timestamp of -60 s was decoded as
2^64 ns - 60 s
which produced timestamps approximately 584 years in the future. The resulting values exceeded the valid pandas datetime range and appeared as NaT.
Summary of Changes
this fixes a bug observed with pre-recording buffers in some CANape blf recordings.
Related Issues / Pull Requests
no issue opened.
Type of Change
Checklist
tox). (blf related ones: )PASS [ 4.004ms] test/logformats_test.py::TestBlfFileFormat::test_append_mode
PASS [ 877.1us] test/logformats_test.py::TestBlfFileFormat::test_can_error_frame_ext
PASS [ 1.020ms] test/logformats_test.py::TestBlfFileFormat::test_can_fd_message
PASS [ 828.9us] test/logformats_test.py::TestBlfFileFormat::test_can_fd_message_64
PASS [ 838.3us] test/logformats_test.py::TestBlfFileFormat::test_can_message
PASS [ 721.1us] test/logformats_test.py::TestBlfFileFormat::test_can_message_2
PASS [ 2.659ms] test/logformats_test.py::TestBlfFileFormat::test_file_like_context_manager
PASS [ 2.611ms] test/logformats_test.py::TestBlfFileFormat::test_file_like_explicit_stop
PASS [ 651.6us] test/logformats_test.py::TestBlfFileFormat::test_issue_1905
PASS [ 2.605ms] test/logformats_test.py::TestBlfFileFormat::test_path_like_context_manager
PASS [ 2.172ms] test/logformats_test.py::TestBlfFileFormat::test_path_like_explicit_stop
PASS [ 355.9us] test/logformats_test.py::TestBlfFileFormat::test_timestamp_to_systemtime
Additional Notes