Skip to content

Instantly share code, notes, and snippets.

@jcstein

jcstein/mamo-1-ln.log

Created March 20, 2025 17:34
Show Gist options
  • Save jcstein/78929128a933f4501d43181d6220b2fb to your computer and use it in GitHub Desktop.
Save jcstein/78929128a933f4501d43181d6220b2fb to your computer and use it in GitHub Desktop.
Download ZIP
Raw
mamo-1-ln.log
2025-03-20T11:46:56.880-0400 INFO header/store store/store.go:367 new head {"height": 53602, "hash": "695304044FDDE658629B8533101697A8D6FEB229763C4E0E2C1A62424CCDFC87"}
2025-03-20T11:46:56.880-0400 INFO header/store store/store.go:367 new head {"height": 53603, "hash": "F223417CD2BE0C3561392BF7FA913AE77F098B116074B662A7EB10B3049E8E10"}
2025-03-20T11:46:56.880-0400 INFO header/sync sync/sync.go:234 finished syncing headers {"from": 53602, "to": 53603, "elapsed time": 0.137241666}
2025-03-20T11:46:56.880-0400 INFO das das/worker.go:95 finished sampling headers {"type": "catchup", "from": 53602, "to": 53603, "errors": 0, "# of headers skipped as outside of sampling window": 0, "finished (s)": 0.137020208}
2025-03-20T11:47:05.565-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53603, "err": "header: not found"}
2025-03-20T11:47:35.812-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53603, "err": "header: not found"}
2025-03-20T11:49:43.683-0400 INFO share/discovery discovery/discovery.go:333 discovered wanted peers {"topic": "/full/v0.1.0", "amount": 5}
2025-03-20T11:49:58.387-0400 INFO header/sync sync/sync.go:216 syncing headers {"from": 53604, "to": 53605}
2025-03-20T11:49:58.387-0400 INFO header/sync sync/sync_head.go:140 new network head {"height": 53605, "hash": "4B27876BB484617F2105861AD370938CDF3CDD3966C8645C510D2E166B18AEA9"}
2025-03-20T11:49:58.387-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53605, "hash": "4B27876BB484617F2105861AD370938CDF3CDD3966C8645C510D2E166B18AEA9", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.000015792}
2025-03-20T11:49:58.522-0400 INFO header/store store/store.go:367 new head {"height": 53604, "hash": "2087EDB4ADB84235EFCEBC8D52604FC00F628E05F0C7851F4E62127C0DBFAB3C"}
2025-03-20T11:49:58.523-0400 INFO header/store store/store.go:367 new head {"height": 53605, "hash": "4B27876BB484617F2105861AD370938CDF3CDD3966C8645C510D2E166B18AEA9"}
2025-03-20T11:49:58.523-0400 INFO header/sync sync/sync.go:234 finished syncing headers {"from": 53604, "to": 53605, "elapsed time": 0.13540675}
2025-03-20T11:49:58.523-0400 INFO das das/worker.go:95 finished sampling headers {"type": "catchup", "from": 53604, "to": 53605, "errors": 0, "# of headers skipped as outside of sampling window": 0, "finished (s)": 0.135305584}
2025-03-20T11:50:11.821-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53605, "err": "header: not found"}
2025-03-20T11:52:19.788-0400 INFO share/discovery discovery/discovery.go:333 discovered wanted peers {"topic": "/full/v0.1.0", "amount": 5}
2025-03-20T11:52:27.653-0400 WARN share/discovery discovery/discovery.go:240 Potentially degraded connectivity, unable to discover the desired amount of /archival/v0.1.0 peers in 5m0s. Number of peers discovered: 0. Required: 5.
2025-03-20T11:52:27.654-0400 INFO pidstore pidstore/pidstore.go:84 Persisted peers successfully {"amount": 6}
2025-03-20T11:52:30.481-0400 INFO header/sync sync/sync.go:216 syncing headers {"from": 53606, "to": 53607}
2025-03-20T11:52:30.480-0400 INFO header/sync sync/sync_head.go:140 new network head {"height": 53607, "hash": "EC60943F35B2F111C0107E143067C6076CAEE618E077490C55537A8C834E84E0"}
2025-03-20T11:52:30.615-0400 INFO header/store store/store.go:367 new head {"height": 53606, "hash": "ACAAA185345D72421E3287FBDCB364EC65E4A8F2FD0912DCA3D970B113C3A49D"}
2025-03-20T11:52:30.616-0400 INFO header/store store/store.go:367 new head {"height": 53607, "hash": "EC60943F35B2F111C0107E143067C6076CAEE618E077490C55537A8C834E84E0"}
2025-03-20T11:52:30.616-0400 INFO header/sync sync/sync.go:234 finished syncing headers {"from": 53606, "to": 53607, "elapsed time": 0.134990584}
2025-03-20T11:52:45.274-0400 INFO header/store store/store.go:367 new head {"height": 53608, "hash": "13D38CC04F41159430FCC9C9C8239082E084B99A6CF149DCE21DED347E6D2F95"}
2025-03-20T11:52:45.275-0400 INFO das das/worker.go:95 finished sampling headers {"type": "catchup", "from": 53606, "to": 53608, "errors": 0, "# of headers skipped as outside of sampling window": 0, "finished (s)": 0.0000685}
2025-03-20T11:52:45.275-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53608, "hash": "13D38CC04F41159430FCC9C9C8239082E084B99A6CF149DCE21DED347E6D2F95", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.000032917}
2025-03-20T11:53:00.706-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53608, "err": "header: not found"}
2025-03-20T11:53:30.990-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53608, "err": "header: not found"}
2025-03-20T11:54:01.214-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53608, "err": "header: not found"}
2025-03-20T11:54:06.200-0400 INFO header/store store/store.go:367 new head {"height": 53609, "hash": "388DF12BB159604B7230FBB4D6B60119EADFDC0E0B18086302B99729854675A1"}
2025-03-20T11:54:06.200-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53609, "hash": "388DF12BB159604B7230FBB4D6B60119EADFDC0E0B18086302B99729854675A1", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.000024458}
2025-03-20T11:55:29.455-0400 INFO share/discovery discovery/discovery.go:333 discovered wanted peers {"topic": "/full/v0.1.0", "amount": 5}
2025-03-20T11:55:41.029-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53609, "err": "header: not found"}
2025-03-20T11:55:48.490-0400 INFO header/store store/store.go:367 new head {"height": 53610, "hash": "324F681E3CE4EEA9F17330B984963ED82A5EFFE5C55368222699D7F02DD09AE3"}
2025-03-20T11:55:48.490-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53610, "hash": "324F681E3CE4EEA9F17330B984963ED82A5EFFE5C55368222699D7F02DD09AE3", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.000024708}
2025-03-20T11:56:11.256-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53610, "err": "header: not found"}
2025-03-20T11:57:07.211-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53610, "err": "header: not found"}
2025-03-20T11:57:20.248-0400 INFO header/store store/store.go:367 new head {"height": 53611, "hash": "C2F934CC62678EAD3DDDEDF54D95D1F8E1BEFB0518870BC57C853EF2786A0232"}
2025-03-20T11:57:20.248-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53611, "hash": "C2F934CC62678EAD3DDDEDF54D95D1F8E1BEFB0518870BC57C853EF2786A0232", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.00002275}
2025-03-20T11:57:45.471-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53611, "err": "header: not found"}
2025-03-20T11:58:13.697-0400 INFO header/store store/store.go:367 new head {"height": 53612, "hash": "0139A1C8A1D2A4ED67A00F5081EB014F476969AA37CEA189B52027A8601F46C3"}
2025-03-20T11:58:13.697-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53612, "hash": "0139A1C8A1D2A4ED67A00F5081EB014F476969AA37CEA189B52027A8601F46C3", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.000024083}
2025-03-20T11:58:15.665-0400 WARN header/p2p p2p/exchange.go:199 verifying head received from tracked peer {"tracked peer": "12D3KooWSJZXE1qGE8yoZBEAEtjt8m4AS3fA7AChuJ9XsgqkDhkB", "height": 53611, "err": "header verification failed: unordered headers: timestamp '2025-03-20T15:55:48Z' < current '2025-03-20T15:57:19Z'"}
2025-03-20T11:58:15.680-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53612, "err": "header: not found"}
2025-03-20T12:15:01.457-0400 INFO share/discovery discovery/discovery.go:333 discovered wanted peers {"topic": "/full/v0.1.0", "amount": 5}
2025-03-20T12:15:19.006-0400 ERROR header/p2p p2p/exchange.go:179 head request to peer failed {"peer": "12D3KooWCMukHJmtehHCx8N3CiXuP8iqJ5tMXGABQep9YcAxFheS", "err": "received a stateless reset"}
2025-03-20T12:15:19.009-0400 INFO header/sync sync/sync_head.go:140 new network head {"height": 53625, "hash": "922423EBF5411F090CB22FFF01D8BE352C6D90C4DB7B3B3FBE25A38B8F0B69F5"}
2025-03-20T12:15:19.009-0400 INFO header/sync sync/sync.go:216 syncing headers {"from": 53613, "to": 53625}
2025-03-20T12:15:19.225-0400 INFO header/store store/store.go:367 new head {"height": 53624, "hash": "8E1A40E2DD75A57C10C7936CA616AFD21403DB73AB5AD5784647D35E36A5F820"}
2025-03-20T12:15:19.225-0400 INFO header/store store/store.go:367 new head {"height": 53625, "hash": "922423EBF5411F090CB22FFF01D8BE352C6D90C4DB7B3B3FBE25A38B8F0B69F5"}
2025-03-20T12:15:19.225-0400 INFO header/sync sync/sync.go:234 finished syncing headers {"from": 53613, "to": 53625, "elapsed time": 0.2149955}
2025-03-20T12:15:27.727-0400 INFO header/store store/store.go:367 new head {"height": 53626, "hash": "BBFB37A871F4CB892E5605D6B22881FDDD509993949BF34B266D07E7ACC40549"}
2025-03-20T12:15:27.727-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53626, "hash": "BBFB37A871F4CB892E5605D6B22881FDDD509993949BF34B266D07E7ACC40549", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.0001055}
2025-03-20T12:15:28.199-0400 INFO das das/worker.go:95 finished sampling headers {"type": "catchup", "from": 53613, "to": 53626, "errors": 0, "# of headers skipped as outside of sampling window": 0, "finished (s)": 0.472247083}
2025-03-20T12:15:43.273-0400 INFO pidstore pidstore/pidstore.go:84 Persisted peers successfully {"amount": 6}
2025-03-20T12:15:43.749-0400 WARN share/discovery discovery/discovery.go:240 Potentially degraded connectivity, unable to discover the desired amount of /archival/v0.1.0 peers in 5m0s. Number of peers discovered: 0. Required: 5.
2025-03-20T12:15:44.618-0400 INFO das das/store.go:60 stored checkpoint to disk: SampleFrom: 53627, NetworkHead: 53626
2025-03-20T12:15:49.210-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53626, "err": "header: not found"}
2025-03-20T12:16:19.432-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53626, "err": "header: not found"}
2025-03-20T12:16:49.650-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53626, "err": "header: not found"}
2025-03-20T12:17:19.889-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53626, "err": "header: not found"}
2025-03-20T12:17:21.283-0400 INFO header/store store/store.go:367 new head {"height": 53627, "hash": "D5F3129A042B434DBBF49A5D7C58F04EB21269035DBD8F782BCCA42CFA4018FB"}
2025-03-20T12:17:21.283-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53627, "hash": "D5F3129A042B434DBBF49A5D7C58F04EB21269035DBD8F782BCCA42CFA4018FB", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.000010875}
2025-03-20T12:17:50.111-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53627, "err": "header: not found"}
2025-03-20T12:18:20.357-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53627, "err": "header: not found"}
2025-03-20T12:18:45.734-0400 INFO header/store store/store.go:367 new head {"height": 53628, "hash": "C50FDEBF90F117629D815C83A6F608180ED4ED93E9E2039136B448945A32AAF4"}
2025-03-20T12:18:45.734-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53628, "hash": "C50FDEBF90F117629D815C83A6F608180ED4ED93E9E2039136B448945A32AAF4", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.000042042}
2025-03-20T12:18:50.586-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53628, "err": "header: not found"}
2025-03-20T12:19:20.811-0400 WARN header/p2p p2p/exchange.go:199 verifying head received from tracked peer {"tracked peer": "12D3KooWKfWxVGHSwxmfvX7dyoke22aCM9bKKFUY4G24ps9Ufu8j", "height": 36160, "err": "header verification failed: unordered headers: timestamp '2025-03-19T23:35:06Z' < current '2025-03-20T16:17:20Z'"}
2025-03-20T12:19:20.811-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53628, "err": "header: not found"}
2025-03-20T12:19:51.030-0400 WARN header/p2p p2p/exchange.go:199 verifying head received from tracked peer {"tracked peer": "12D3KooWKfWxVGHSwxmfvX7dyoke22aCM9bKKFUY4G24ps9Ufu8j", "height": 45376, "err": "header verification failed: unordered headers: timestamp '2025-03-20T08:06:23Z' < current '2025-03-20T16:17:20Z'"}
2025-03-20T12:19:51.030-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53628, "err": "header: not found"}
2025-03-20T12:19:58.429-0400 INFO header/store store/store.go:367 new head {"height": 53629, "hash": "8E731D85F3BFA3D229A4F34E270CE00099DDEFC1B9CC5AEAD4835BD8ABE99D81"}
2025-03-20T12:19:58.430-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53629, "hash": "8E731D85F3BFA3D229A4F34E270CE00099DDEFC1B9CC5AEAD4835BD8ABE99D81", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.00002}
2025-03-20T12:20:21.693-0400 WARN header/p2p p2p/exchange.go:199 verifying head received from tracked peer {"tracked peer": "12D3KooWKfWxVGHSwxmfvX7dyoke22aCM9bKKFUY4G24ps9Ufu8j", "height": 49984, "err": "header verification failed: unordered headers: timestamp '2025-03-20T12:19:58Z' < current '2025-03-20T16:18:45Z'"}
2025-03-20T12:20:21.694-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53629, "err": "header: not found"}
2025-03-20T12:20:29.228-0400 WARN header/p2p p2p/server.go:157 server: writing header to stream {"err": "stream reset (remote): code: 0x0: transport error: stream 124 canceled by remote with error code 0"}
2025-03-20T12:20:43.275-0400 INFO pidstore pidstore/pidstore.go:84 Persisted peers successfully {"amount": 8}
2025-03-20T12:20:43.411-0400 WARN share/discovery discovery/discovery.go:240 Potentially degraded connectivity, unable to discover the desired amount of /archival/v0.1.0 peers in 5m0s. Number of peers discovered: 0. Required: 5.
2025-03-20T12:20:51.892-0400 WARN header/p2p p2p/exchange.go:199 verifying head received from tracked peer {"tracked peer": "12D3KooWKfWxVGHSwxmfvX7dyoke22aCM9bKKFUY4G24ps9Ufu8j", "height": 52032, "err": "header verification failed: unordered headers: timestamp '2025-03-20T14:04:25Z' < current '2025-03-20T16:18:45Z'"}
2025-03-20T12:20:51.892-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53629, "err": "header: not found"}
2025-03-20T12:20:52.673-0400 INFO header/store store/store.go:367 new head {"height": 53630, "hash": "920EDA066FA7BC0C4B9F736B005082AFD44EF626EA344B892C291CC217B23A5C"}
2025-03-20T12:20:52.673-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53630, "hash": "920EDA066FA7BC0C4B9F736B005082AFD44EF626EA344B892C291CC217B23A5C", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.000026208}
2025-03-20T12:21:03.486-0400 WARN header/p2p p2p/server.go:157 server: writing header to stream {"err": "stream reset (remote): code: 0x0: transport error: stream 144 canceled by remote with error code 0"}
2025-03-20T12:21:11.736-0400 WARN header/p2p p2p/server.go:157 server: writing header to stream {"err": "stream reset (remote): code: 0x0: transport error: stream 148 canceled by remote with error code 0"}
2025-03-20T12:21:21.678-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53630, "err": "header: not found"}
2025-03-20T12:21:51.938-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53630, "err": "header: not found"}
2025-03-20T12:27:40.613-0400 INFO canonical-log swarm/swarm_dial.go:620 CANONICAL_PEER_STATUS: peer=12D3KooWMtTXAfjXCVNUTfB8n9zdL8XZBq1MteCR7Uoyo348i1xJ addr=/ip4/51.15.91.195/udp/2121/quic-v1 sample_rate=100 connection_status="established" dir="outbound"
2025-03-20T12:27:40.866-0400 INFO share/discovery discovery/discovery.go:333 discovered wanted peers {"topic": "/full/v0.1.0", "amount": 5}
2025-03-20T12:27:50.086-0400 INFO header/sync sync/sync.go:216 syncing headers {"from": 53631, "to": 53636}
2025-03-20T12:27:50.086-0400 INFO header/sync sync/sync_head.go:140 new network head {"height": 53636, "hash": "15D0F3543E8CBA0F31710CDF87025568BE6869B09649132B045ACFBEEA322AA2"}
2025-03-20T12:27:50.087-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53636, "hash": "15D0F3543E8CBA0F31710CDF87025568BE6869B09649132B045ACFBEEA322AA2", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.000018917}
2025-03-20T12:27:50.235-0400 INFO header/store store/store.go:367 new head {"height": 53635, "hash": "02C1C7BDBFA7C21C1397D35A203BB1261BC8CB1F30F376A36BE4B1CB4FED7854"}
2025-03-20T12:27:50.235-0400 INFO header/store store/store.go:367 new head {"height": 53636, "hash": "15D0F3543E8CBA0F31710CDF87025568BE6869B09649132B045ACFBEEA322AA2"}
2025-03-20T12:27:50.235-0400 INFO header/sync sync/sync.go:234 finished syncing headers {"from": 53631, "to": 53636, "elapsed time": 0.148652917}
2025-03-20T12:27:50.235-0400 INFO das das/worker.go:95 finished sampling headers {"type": "catchup", "from": 53631, "to": 53636, "errors": 0, "# of headers skipped as outside of sampling window": 0, "finished (s)": 0.148511166}
2025-03-20T12:27:55.432-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53636, "err": "header: not found"}
2025-03-20T12:28:25.690-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53636, "err": "header: not found"}
2025-03-20T12:37:38.200-0400 INFO share/discovery discovery/discovery.go:333 discovered wanted peers {"topic": "/full/v0.1.0", "amount": 5}
2025-03-20T12:38:06.116-0400 INFO header/sync sync/sync.go:216 syncing headers {"from": 53637, "to": 53644}
2025-03-20T12:38:06.116-0400 INFO header/sync sync/sync_head.go:140 new network head {"height": 53644, "hash": "53BAE1525A11C364849E46F17C2478A2D315BD39E5030EC0B455FF6E66A43C47"}
2025-03-20T12:38:06.302-0400 INFO header/store store/store.go:367 new head {"height": 53643, "hash": "35D23BD107783799F2578422AACABC3A079BDEFE0430FF6D4CD6603761436923"}
2025-03-20T12:38:06.302-0400 INFO header/store store/store.go:367 new head {"height": 53644, "hash": "53BAE1525A11C364849E46F17C2478A2D315BD39E5030EC0B455FF6E66A43C47"}
2025-03-20T12:38:06.302-0400 INFO header/sync sync/sync.go:234 finished syncing headers {"from": 53637, "to": 53644, "elapsed time": 0.186503959}
2025-03-20T12:38:34.807-0400 INFO header/store store/store.go:367 new head {"height": 53645, "hash": "871A42645E48367C823BC2BE6E33AF4DFBBFFB04F6DF8BAF5715CB6F3DCC5541"}
2025-03-20T12:38:34.807-0400 INFO das das/worker.go:95 finished sampling headers {"type": "catchup", "from": 53637, "to": 53645, "errors": 0, "# of headers skipped as outside of sampling window": 0, "finished (s)": 0.000149375}
2025-03-20T12:38:34.807-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53645, "hash": "871A42645E48367C823BC2BE6E33AF4DFBBFFB04F6DF8BAF5715CB6F3DCC5541", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.000074833}
2025-03-20T12:38:36.191-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53645, "err": "header: not found"}
2025-03-20T12:54:21.357-0400 ERROR header/p2p p2p/exchange.go:179 head request to peer failed {"peer": "12D3KooWQAm6jsEfsAZ8QSs2FRDyNyQBfDHkiC9WCSpyMG58Xc5o", "err": "received a stateless reset"}
2025-03-20T12:54:21.358-0400 ERROR header/p2p p2p/exchange.go:179 head request to peer failed {"peer": "12D3KooWMtTXAfjXCVNUTfB8n9zdL8XZBq1MteCR7Uoyo348i1xJ", "err": "received a stateless reset"}
2025-03-20T12:54:21.359-0400 ERROR header/p2p p2p/exchange.go:179 head request to peer failed {"peer": "12D3KooWNc3hDtzLvyKj8xbcE3SFMRy4uX5EojCScCuqYRrz4tzS", "err": "received a stateless reset"}
2025-03-20T12:54:22.339-0400 ERROR header/p2p p2p/exchange.go:179 head request to peer failed {"peer": "12D3KooWCMukHJmtehHCx8N3CiXuP8iqJ5tMXGABQep9YcAxFheS", "err": "deadline exceeded"}
2025-03-20T12:54:22.339-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53645, "err": "header: not found"}
2025-03-20T12:54:22.418-0400 INFO share/discovery discovery/discovery.go:333 discovered wanted peers {"topic": "/full/v0.1.0", "amount": 5}
2025-03-20T12:54:50.901-0400 INFO header/sync sync/sync.go:216 syncing headers {"from": 53646, "to": 53658}
2025-03-20T12:54:50.901-0400 INFO header/sync sync/sync_head.go:140 new network head {"height": 53658, "hash": "0599019AEB72C471CB36A86AC2FDE58B681597E46ECEC12636FE47C24D4B7F0D"}
2025-03-20T12:54:51.116-0400 INFO header/store store/store.go:367 new head {"height": 53657, "hash": "6E0E6C6C268B0A12A48F3393D115A3A04CD14A799423C5280680561DBC75ED18"}
2025-03-20T12:54:51.116-0400 INFO header/store store/store.go:367 new head {"height": 53658, "hash": "0599019AEB72C471CB36A86AC2FDE58B681597E46ECEC12636FE47C24D4B7F0D"}
2025-03-20T12:54:51.116-0400 INFO header/sync sync/sync.go:234 finished syncing headers {"from": 53646, "to": 53658, "elapsed time": 0.214778875}
2025-03-20T12:55:10.534-0400 INFO header/store store/store.go:367 new head {"height": 53659, "hash": "E8A6F82F5D0B39697882786ACB28BC82F6E488686853ED7D1802F668A6898361"}
2025-03-20T12:55:10.534-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53659, "hash": "E8A6F82F5D0B39697882786ACB28BC82F6E488686853ED7D1802F668A6898361", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.000031}
2025-03-20T12:55:10.534-0400 INFO das das/worker.go:95 finished sampling headers {"type": "catchup", "from": 53646, "to": 53659, "errors": 0, "# of headers skipped as outside of sampling window": 0, "finished (s)": 0.000173542}
2025-03-20T12:55:21.163-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53659, "err": "header: not found"}
2025-03-20T12:55:40.196-0400 INFO pidstore pidstore/pidstore.go:84 Persisted peers successfully {"amount": 6}
2025-03-20T12:55:40.442-0400 WARN share/discovery discovery/discovery.go:240 Potentially degraded connectivity, unable to discover the desired amount of /archival/v0.1.0 peers in 5m0s. Number of peers discovered: 0. Required: 5.
2025-03-20T12:55:41.541-0400 INFO das das/store.go:60 stored checkpoint to disk: SampleFrom: 53660, NetworkHead: 53659
2025-03-20T12:55:51.392-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53659, "err": "header: not found"}
2025-03-20T12:56:21.618-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53659, "err": "header: not found"}
2025-03-20T12:57:41.263-0400 INFO share/discovery discovery/discovery.go:333 discovered wanted peers {"topic": "/full/v0.1.0", "amount": 5}
2025-03-20T12:57:45.249-0400 INFO share/discovery discovery/discovery.go:333 discovered wanted peers {"topic": "/full/v0.1.0", "amount": 5}
2025-03-20T12:57:49.277-0400 INFO header/sync sync/sync.go:216 syncing headers {"from": 53660, "to": 53661}
2025-03-20T12:57:49.277-0400 INFO header/sync sync/sync_head.go:140 new network head {"height": 53661, "hash": "4B8C11ADB937DE862E3D6E678AAC1C39D107CF2F7715EF384B45059775D77B88"}
2025-03-20T12:57:49.277-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53661, "hash": "4B8C11ADB937DE862E3D6E678AAC1C39D107CF2F7715EF384B45059775D77B88", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.000016125}
2025-03-20T12:57:49.453-0400 INFO header/store store/store.go:367 new head {"height": 53660, "hash": "FF53060C18623ADCA86093646C94481D27019E1433CBCE5D3A0CF63ACCB91FD3"}
2025-03-20T12:57:49.453-0400 INFO header/store store/store.go:367 new head {"height": 53661, "hash": "4B8C11ADB937DE862E3D6E678AAC1C39D107CF2F7715EF384B45059775D77B88"}
2025-03-20T12:57:49.453-0400 INFO header/sync sync/sync.go:234 finished syncing headers {"from": 53660, "to": 53661, "elapsed time": 0.175651375}
2025-03-20T12:57:49.453-0400 INFO das das/worker.go:95 finished sampling headers {"type": "catchup", "from": 53660, "to": 53661, "errors": 0, "# of headers skipped as outside of sampling window": 0, "finished (s)": 0.175547458}
2025-03-20T12:57:57.819-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53661, "err": "header: not found"}
2025-03-20T12:58:28.053-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53661, "err": "header: not found"}
2025-03-20T12:58:54.129-0400 INFO header/store store/store.go:367 new head {"height": 53662, "hash": "F4263C49D85619B87E0928036E3789386B4804DBA871DAD3D520931A76763FD2"}
2025-03-20T12:58:54.130-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53662, "hash": "F4263C49D85619B87E0928036E3789386B4804DBA871DAD3D520931A76763FD2", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.00001375}
2025-03-20T12:58:58.267-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53662, "err": "header: not found"}
2025-03-20T12:59:28.501-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53662, "err": "header: not found"}
2025-03-20T12:59:56.440-0400 INFO header/store store/store.go:367 new head {"height": 53663, "hash": "30EE24ED0A17AA1E07B666226D686E59E059D252E1429FB6F54EC13BBD6F112B"}
2025-03-20T12:59:56.440-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53663, "hash": "30EE24ED0A17AA1E07B666226D686E59E059D252E1429FB6F54EC13BBD6F112B", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.000019917}
2025-03-20T12:59:58.700-0400 WARN header/p2p p2p/exchange.go:199 verifying head received from tracked peer {"tracked peer": "12D3KooWSJZXE1qGE8yoZBEAEtjt8m4AS3fA7AChuJ9XsgqkDhkB", "height": 53662, "err": "header verification failed: unordered headers: timestamp '2025-03-20T16:57:48Z' < current '2025-03-20T16:58:53Z'"}
2025-03-20T12:59:58.700-0400 WARN header/p2p p2p/exchange.go:199 verifying head received from tracked peer {"tracked peer": "12D3KooWMtTXAfjXCVNUTfB8n9zdL8XZBq1MteCR7Uoyo348i1xJ", "height": 53661, "err": "header verification failed: unordered headers: timestamp '2025-03-20T16:56:34Z' < current '2025-03-20T16:58:53Z'"}
2025-03-20T12:59:58.723-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53663, "err": "header: not found"}
2025-03-20T13:00:28.871-0400 WARN header/p2p p2p/exchange.go:199 verifying head received from tracked peer {"tracked peer": "12D3KooWMtTXAfjXCVNUTfB8n9zdL8XZBq1MteCR7Uoyo348i1xJ", "height": 53661, "err": "header verification failed: unordered headers: timestamp '2025-03-20T16:56:34Z' < current '2025-03-20T16:58:53Z'"}
2025-03-20T13:00:28.893-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53663, "err": "header: not found"}
2025-03-20T13:00:57.340-0400 INFO header/store store/store.go:367 new head {"height": 53664, "hash": "9B79ACE067AEA25DBBCE2892FACF55E9E61D3FD9BE437CC33880BF52D4CAD0F8"}
2025-03-20T13:00:57.340-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53664, "hash": "9B79ACE067AEA25DBBCE2892FACF55E9E61D3FD9BE437CC33880BF52D4CAD0F8", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.000021458}
2025-03-20T13:00:59.076-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53664, "err": "header: not found"}
2025-03-20T13:01:29.279-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53664, "err": "header: not found"}
2025-03-20T13:01:46.149-0400 INFO pidstore pidstore/pidstore.go:84 Persisted peers successfully {"amount": 7}
2025-03-20T13:01:46.395-0400 WARN share/discovery discovery/discovery.go:240 Potentially degraded connectivity, unable to discover the desired amount of /archival/v0.1.0 peers in 5m0s. Number of peers discovered: 0. Required: 5.
2025-03-20T13:01:59.468-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53664, "err": "header: not found"}
2025-03-20T13:02:11.634-0400 INFO header/store store/store.go:367 new head {"height": 53665, "hash": "07C49644F05AADBDCC566A4035EEED46311494B750D55C4075724BCE23F8F98F"}
2025-03-20T13:02:11.634-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53665, "hash": "07C49644F05AADBDCC566A4035EEED46311494B750D55C4075724BCE23F8F98F", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.00001575}
2025-03-20T13:02:29.665-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53665, "err": "header: not found"}
2025-03-20T13:02:59.887-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53665, "err": "header: not found"}
2025-03-20T13:03:16.481-0400 INFO header/store store/store.go:367 new head {"height": 53666, "hash": "C11290D073168DE47215994EEA64BB61F9ABD57D1E4A8718EFBF8AFD6E34F781"}
2025-03-20T13:03:16.482-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53666, "hash": "C11290D073168DE47215994EEA64BB61F9ABD57D1E4A8718EFBF8AFD6E34F781", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.000020208}
2025-03-20T13:03:30.119-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53666, "err": "header: not found"}
2025-03-20T13:04:00.309-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53666, "err": "header: not found"}
2025-03-20T13:04:25.280-0400 INFO header/store store/store.go:367 new head {"height": 53667, "hash": "7C5358EE1DEF9524C834EE37466F0611B6C08956C0B993EA94D1A1CD93F0CD92"}
2025-03-20T13:04:25.280-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53667, "hash": "7C5358EE1DEF9524C834EE37466F0611B6C08956C0B993EA94D1A1CD93F0CD92", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.000023083}
2025-03-20T13:04:30.532-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53667, "err": "header: not found"}
2025-03-20T13:05:00.747-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53667, "err": "header: not found"}
2025-03-20T13:05:12.542-0400 INFO header/store store/store.go:367 new head {"height": 53668, "hash": "79AB4A52BE854C744AE5204603403F4476B52A5977221C013AAD63B9E2D58BA1"}
2025-03-20T13:05:12.542-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53668, "hash": "79AB4A52BE854C744AE5204603403F4476B52A5977221C013AAD63B9E2D58BA1", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.000021584}
2025-03-20T13:05:30.938-0400 WARN header/p2p p2p/exchange.go:199 verifying head received from tracked peer {"tracked peer": "12D3KooWMtTXAfjXCVNUTfB8n9zdL8XZBq1MteCR7Uoyo348i1xJ", "height": 53667, "err": "header verification failed: unordered headers: timestamp '2025-03-20T17:03:16Z' < current '2025-03-20T17:04:24Z'"}
2025-03-20T13:05:30.939-0400 WARN header/p2p p2p/exchange.go:199 verifying head received from tracked peer {"tracked peer": "12D3KooWSJZXE1qGE8yoZBEAEtjt8m4AS3fA7AChuJ9XsgqkDhkB", "height": 53667, "err": "header verification failed: unordered headers: timestamp '2025-03-20T17:03:16Z' < current '2025-03-20T17:04:24Z'"}
2025-03-20T13:05:30.962-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53668, "err": "header: not found"}
2025-03-20T13:06:01.180-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53668, "err": "header: not found"}
2025-03-20T13:06:24.355-0400 INFO header/store store/store.go:367 new head {"height": 53669, "hash": "C7B50E14CE798AB273C0A13B84C3D61DE565EE2712C0E7BF7A436B5B19B030E2"}
2025-03-20T13:06:24.356-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53669, "hash": "C7B50E14CE798AB273C0A13B84C3D61DE565EE2712C0E7BF7A436B5B19B030E2", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.000018125}
2025-03-20T13:06:31.411-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53669, "err": "header: not found"}
2025-03-20T13:06:46.150-0400 INFO pidstore pidstore/pidstore.go:84 Persisted peers successfully {"amount": 7}
2025-03-20T13:06:46.394-0400 WARN share/discovery discovery/discovery.go:240 Potentially degraded connectivity, unable to discover the desired amount of /archival/v0.1.0 peers in 5m0s. Number of peers discovered: 0. Required: 5.
2025-03-20T13:06:47.496-0400 INFO das das/store.go:60 stored checkpoint to disk: SampleFrom: 53670, NetworkHead: 53669
2025-03-20T13:07:01.624-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53669, "err": "header: not found"}
2025-03-20T13:07:31.866-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53669, "err": "header: not found"}
2025-03-20T13:07:58.391-0400 INFO header/store store/store.go:367 new head {"height": 53670, "hash": "36BEF2D0420E1C18F928111CD6D2AC5F06469D3BCD8C52CCEB5D8627FB03A107"}
2025-03-20T13:07:58.391-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53670, "hash": "36BEF2D0420E1C18F928111CD6D2AC5F06469D3BCD8C52CCEB5D8627FB03A107", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.000018709}
2025-03-20T13:08:02.069-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53670, "err": "header: not found"}
2025-03-20T13:08:32.269-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53670, "err": "header: not found"}
2025-03-20T13:09:02.445-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53670, "err": "header: not found"}
2025-03-20T13:09:12.238-0400 INFO header/store store/store.go:367 new head {"height": 53671, "hash": "7DD8C2FD7E6DB707EC22B31D5247B58AC76326796F0E844BA13B987BA6CA61FE"}
2025-03-20T13:09:12.238-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53671, "hash": "7DD8C2FD7E6DB707EC22B31D5247B58AC76326796F0E844BA13B987BA6CA61FE", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.000015666}
2025-03-20T13:09:32.647-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53671, "err": "header: not found"}
2025-03-20T13:10:02.875-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53671, "err": "header: not found"}
2025-03-20T13:10:33.113-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53671, "err": "header: not found"}
2025-03-20T13:10:36.108-0400 INFO header/store store/store.go:367 new head {"height": 53672, "hash": "9E73557D1AEDAC76B8C819BA203581247A417EF6DF7DB3D0421871B9F18A629D"}
2025-03-20T13:10:36.109-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53672, "hash": "9E73557D1AEDAC76B8C819BA203581247A417EF6DF7DB3D0421871B9F18A629D", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.000015584}
2025-03-20T13:11:03.307-0400 WARN header/p2p p2p/exchange.go:199 verifying head received from tracked peer {"tracked peer": "12D3KooWSJZXE1qGE8yoZBEAEtjt8m4AS3fA7AChuJ9XsgqkDhkB", "height": 53671, "err": "header verification failed: unordered headers: timestamp '2025-03-20T17:07:58Z' < current '2025-03-20T17:09:11Z'"}
2025-03-20T13:11:03.327-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53672, "err": "header: not found"}
2025-03-20T13:11:33.503-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53672, "err": "header: not found"}
2025-03-20T13:11:43.395-0400 INFO header/store store/store.go:367 new head {"height": 53673, "hash": "8C4EDFA518A58C2EB5E3DE09BC73EF69FEA2B72A7F01983C34E33DC4B3035C01"}
2025-03-20T13:11:43.395-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53673, "hash": "8C4EDFA518A58C2EB5E3DE09BC73EF69FEA2B72A7F01983C34E33DC4B3035C01", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.00001675}
2025-03-20T13:11:46.151-0400 INFO pidstore pidstore/pidstore.go:84 Persisted peers successfully {"amount": 7}
2025-03-20T13:11:46.281-0400 WARN share/discovery discovery/discovery.go:240 Potentially degraded connectivity, unable to discover the desired amount of /archival/v0.1.0 peers in 5m0s. Number of peers discovered: 0. Required: 5.
2025-03-20T13:12:03.722-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53673, "err": "header: not found"}
2025-03-20T13:12:33.924-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53673, "err": "header: not found"}
2025-03-20T13:12:50.843-0400 INFO canonical-log tls/transport.go:96 CANONICAL_PEER_STATUS: peer= addr=/ip4/151.115.48.151/tcp/52104 sample_rate=100 handshake_failure="tls" err="read tcp4 10.0.0.191:2121->151.115.48.151:52104: read: connection reset by peer"
2025-03-20T13:13:04.131-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53673, "err": "header: not found"}
2025-03-20T13:13:34.366-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53673, "err": "header: not found"}
2025-03-20T13:13:38.651-0400 INFO header/store store/store.go:367 new head {"height": 53674, "hash": "B9B4EF84967D76B171446F2AE99C2A78887DF9AA65E62D65384861952B072642"}
2025-03-20T13:13:38.651-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53674, "hash": "B9B4EF84967D76B171446F2AE99C2A78887DF9AA65E62D65384861952B072642", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.000024125}
2025-03-20T13:14:04.532-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53674, "err": "header: not found"}
2025-03-20T13:14:34.741-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53674, "err": "header: not found"}
2025-03-20T13:15:04.674-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53674, "err": "header: not found"}
2025-03-20T13:15:35.325-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53674, "err": "header: not found"}
2025-03-20T13:15:38.506-0400 INFO header/store store/store.go:367 new head {"height": 53675, "hash": "44116B4D1C047721EBE447F16F0361468CEAEC117F6E1D0D52356C5EF36D490F"}
2025-03-20T13:15:38.507-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53675, "hash": "44116B4D1C047721EBE447F16F0361468CEAEC117F6E1D0D52356C5EF36D490F", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.000025083}
2025-03-20T13:16:05.502-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53675, "err": "header: not found"}
2025-03-20T13:16:35.672-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53675, "err": "header: not found"}
2025-03-20T13:16:46.150-0400 INFO pidstore pidstore/pidstore.go:84 Persisted peers successfully {"amount": 7}
2025-03-20T13:16:46.389-0400 WARN share/discovery discovery/discovery.go:240 Potentially degraded connectivity, unable to discover the desired amount of /archival/v0.1.0 peers in 5m0s. Number of peers discovered: 0. Required: 5.
2025-03-20T13:16:47.495-0400 INFO das das/store.go:60 stored checkpoint to disk: SampleFrom: 53676, NetworkHead: 53675
2025-03-20T13:17:05.880-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53675, "err": "header: not found"}
2025-03-20T13:17:36.073-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53675, "err": "header: not found"}
2025-03-20T13:18:06.247-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53675, "err": "header: not found"}
2025-03-20T13:18:36.397-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53675, "err": "header: not found"}
2025-03-20T13:18:49.290-0400 INFO header/store store/store.go:367 new head {"height": 53676, "hash": "91B2EAA80D96BD6B06E0AC327E025D5B4693274CAD84EF885B55AD410745433B"}
2025-03-20T13:18:49.290-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53676, "hash": "91B2EAA80D96BD6B06E0AC327E025D5B4693274CAD84EF885B55AD410745433B", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.000018625}
2025-03-20T13:19:06.582-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53676, "err": "header: not found"}
2025-03-20T13:19:36.734-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53676, "err": "header: not found"}
2025-03-20T13:20:07.100-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53676, "err": "header: not found"}
2025-03-20T13:20:37.578-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53676, "err": "header: not found"}
2025-03-20T13:21:07.789-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53676, "err": "header: not found"}
2025-03-20T13:21:38.055-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53676, "err": "header: not found"}
2025-03-20T13:21:46.150-0400 INFO pidstore pidstore/pidstore.go:84 Persisted peers successfully {"amount": 7}
2025-03-20T13:21:46.280-0400 WARN share/discovery discovery/discovery.go:240 Potentially degraded connectivity, unable to discover the desired amount of /archival/v0.1.0 peers in 5m0s. Number of peers discovered: 0. Required: 5.
2025-03-20T13:22:02.875-0400 INFO header/store store/store.go:367 new head {"height": 53677, "hash": "35401923FFE79369AF4E918E2E61D55CDA043EE7F2161800330D8570EF37ED8F"}
2025-03-20T13:22:02.876-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53677, "hash": "35401923FFE79369AF4E918E2E61D55CDA043EE7F2161800330D8570EF37ED8F", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.0000155}
2025-03-20T13:22:08.054-0400 WARN header/p2p p2p/exchange.go:199 verifying head received from tracked peer {"tracked peer": "12D3KooWMtTXAfjXCVNUTfB8n9zdL8XZBq1MteCR7Uoyo348i1xJ", "height": 53676, "err": "header verification failed: unordered headers: timestamp '2025-03-20T17:15:38Z' < current '2025-03-20T17:18:48Z'"}
2025-03-20T13:22:08.071-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53677, "err": "header: not found"}
2025-03-20T13:22:38.068-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53677, "err": "header: not found"}
2025-03-20T13:23:08.059-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53677, "err": "header: not found"}
2025-03-20T13:23:29.972-0400 INFO header/store store/store.go:367 new head {"height": 53678, "hash": "42E678AAA6F11835DC6D5CA03E1B95D1CF7BCE948EE243746D8E162BCD7960A1"}
2025-03-20T13:23:29.973-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53678, "hash": "42E678AAA6F11835DC6D5CA03E1B95D1CF7BCE948EE243746D8E162BCD7960A1", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.000010375}
2025-03-20T13:23:39.032-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53678, "err": "header: not found"}
2025-03-20T13:24:09.260-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53678, "err": "header: not found"}
2025-03-20T13:24:22.038-0400 INFO header/store store/store.go:367 new head {"height": 53679, "hash": "327E5BC95A091CBCC75D137595D83209E22F13326AD6DE8A9D0C30FE1C84D501"}
2025-03-20T13:24:22.038-0400 INFO das das/worker.go:164 sampled header {"type": "recent", "height": 53679, "hash": "327E5BC95A091CBCC75D137595D83209E22F13326AD6DE8A9D0C30FE1C84D501", "EDS square width": 2, "data root": "3D96B7D238E7E0456F6AF8E7CDF0A67BD6CF9C2089ECB559C659DCAA1F880353", "finished (s)": 0.000013458}
2025-03-20T13:24:39.424-0400 WARN header/sync sync/sync_head.go:46 failed to get recent head, returning current subjective {"sbjHead": 53679, "err": "header: not found"}
@jcstein
Copy link
Author

jcstein commented Mar 20, 2025 •
edited
Loading

Generated by claude.ai

Block Heights and Timestamps (in Minutes)

Block Information with Time in Minutes

Block Height Time (UTC) Minutes From Start Minutes Since Previous Block
53602 15:46:56 0.00 -
53603 15:46:56 0.00 0.00
53604 15:49:58 3.03 3.03
53605 15:49:58 3.03 0.00
53606 15:52:30 5.56 2.53
53607 15:52:30 5.56 0.00
53608 15:52:45 5.81 0.24
53609 15:54:06 7.16 1.35
53610 15:55:48 8.86 1.70
53611 15:57:20 10.39 1.53
53612 15:58:13 11.28 0.89
53624 16:15:19 28.37 17.09
53625 16:15:19 28.37 0.00
53626 16:15:27 28.51 0.14
53627 16:17:21 30.41 1.89
53628 16:18:45 31.81 1.41
53629 16:19:58 33.03 1.21
53630 16:20:52 33.93 0.90
53635 16:27:50 40.89 6.96
53636 16:27:50 40.89 0.00
53643 16:38:06 51.16 10.27
53644 16:38:06 51.16 0.00
53645 16:38:34 51.63 0.48
53657 16:54:51 67.90 16.27
53658 16:54:51 67.90 0.00
53659 16:55:10 68.23 0.32
53660 16:57:49 70.88 2.65
53661 16:57:49 70.88 0.00
53662 16:58:54 71.95 1.08
53663 16:59:56 72.99 1.04
53664 17:00:57 74.01 1.01
53665 17:02:11 75.25 1.24
53666 17:03:16 76.33 1.08
53667 17:04:25 77.47 1.15
53668 17:05:12 78.26 0.79
53669 17:06:24 79.46 1.20
53670 17:07:58 81.03 1.57
53671 17:09:12 82.26 1.23
53672 17:10:36 83.65 1.40
53673 17:11:43 84.78 1.12
53674 17:13:38 86.70 1.92
53675 17:15:38 88.69 2.00
53676 17:18:49 91.87 3.18
53677 17:22:02 95.10 3.23
53678 17:23:29 96.55 1.45
53679 17:24:22 97.42 0.87

Summary Statistics

  • Total time span: 97.42 minutes (1 hour, 37 minutes, 25 seconds)
  • Total blocks: 46
  • Average time between blocks: 2.16 minutes

Notable Time Gaps

  • Largest gap: 17.09 minutes between blocks 53612 and 53624
  • Other significant gaps:
    • 16.27 minutes between blocks 53645 and 53657
    • 10.27 minutes between blocks 53636 and 53643
    • 6.96 minutes between blocks 53630 and 53635

Block Production Patterns

The block production appears to have several patterns:

  1. Many blocks are produced in pairs with the same timestamp (e.g., 53602/53603, 53606/53607)
  2. Normal block production seems to be 1-2 minutes per block during stable periods
  3. Several larger gaps indicate possible network issues or synchronization events
  4. The most recent blocks (53676-53679) show slightly longer intervals (1.5-3.2 minutes)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment