Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure

Orbiter Check History

Showing 50 of 834227 reports (Page 1779 of 16685)
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................]
duration:
5.687166458s
trace:
https://jaeger.dev.storj.io/trace/f253484fd40b3cf
FAIL (2026-05-13 08:39:33 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
52.886159919s) [upload=288931]: Starting upload
May 13 08:23:42.365411428 (301h35m52.886235173s) [upload=288931]: Done starting upload
May 13 08:23:42.365500037 (301h35m52.886323782s) [upload=288931]: Uploading segments...
May 13 08:23:42.365647579 (301h35m52.886471334s) [upload=288931]: Got next segment. Inline: false
May 13 08:23:42.365683176 (301h35m52.886506911s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 13 08:23:42.365687885 (301h35m52.88651163s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 13 08:23:42.598383896 (301h35m53.119207641s) [seg_pos={0 0}, upload=288931]: Begin upload segment...
May 13 08:23:42.598464609 (301h35m53.119288344s) [seg_pos={0 0}, upload=288931]: Done begin upload segment.
May 13 08:23:42.598489827 (301h35m53.119313562s) [noise=true , address=145.79.227.4:4185    , piece_id=73P5QCJDK5IESMP53DQNMZHRSOHFDVYXEM6AWW7CWR7PDDKH4EXQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.598584227 (301h35m53.119407972s) [noise=true , address=77.90.31.15:4701     , piece_id=XQI7GRPTXPE35WKWKEQX72RSEQ74NMF7645CXJA2E5FKWWKHEI2A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599099290 (301h35m53.119923035s) [noise=true , address=145.79.227.52:4447   , piece_id=XTJJIUVCL2WH5TJ4CZRJAUETVQIYYB55KALFX46YG2FPOMLLOPUA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599448427 (301h35m53.120272172s) [noise=true , address=145.79.227.3:4152    , piece_id=BZGI2SX3KLTQ5L74GQTXWJ3MXN5KLLFZKMPES5FRGGCLUD7XXV3A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599785570 (301h35m53.120609315s) [noise=true , address=145.79.227.7:4297    , piece_id=W2ZWVO5KC5ZJMC5LZKQ3ZBGEOBGE2Z7DN6BLRZCGPJ2LWDFEIAGA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600089149 (301h35m53.120912894s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600393219 (301h35m53.121216964s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600679686 (301h35m53.121503431s) [noise=true , address=145.79.227.6:4263    , piece_id=MPF6TBBYJ3IQVVBMS36BIV6PAITZ25AYM3TJAI2KO7XBZB2FN7WQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.601010748 (301h35m53.121834493s) [noise=true , address=145.79.227.26:4654   , piece_id=OYHDYSBP5LMUP6ZQONZNELXVOY53ZSJMG33ONRQLLFGYPFKR3P5Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.601837916 (301h35m53.122661651s) [noise=true , address=145.79.227.39:4017   , piece_id=H2XFKXRSXEAG4WM7XDSYFLSHUOECVFXR47PBYVHW4DZLKJXWLU5A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.602900945 (301h35m53.12372469s) [noise=true , address=145.79.227.45:4233   , piece_id=25YQOKEQLNYGD2DZBPWQD5HMUKBXEA2YMYVLRZWQVHCPXWKFOA3Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.603751337 (301h35m53.124575072s) [noise=true , address=145.79.227.25:4619   , piece_id=3EFGZMPN4XP4F4U5OKK5VIYD44FT3WXQB73NZQL4LFYVQMTSUIVA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.604687004 (301h35m53.125510749s) [noise=true , address=145.79.227.13:4804   , piece_id=GAB5UUUHLI3P6YSFVQ6HHBTPCTLQOPE6THKSHKZPSAHLI7G2QFHA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.605665119 (301h35m53.126488864s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.606795627 (301h35m53.127619372s) [noise=true , address=145.79.227.50:4374   , piece_id=VU7FGWLJIQ6BJWLWR4V3FZ2OOKWZ4XUXXIK6UTIOXG4JQRS5QAVQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.607929281 (301h35m53.128753026s) [noise=true , address=145.79.227.31:4798   , piece_id=2ZEX5FUWBC2NYJJCGZ5UMPBGKKXNGXFH27CMWB7I55BWAQ4X33XQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:44.318577649 (301h35m54.839401394s) [upload=288931]: Next returned nil segment
May 13 08:23:44.318598147 (301h35m54.839421892s) [upload=288931]: Waiting for error group managing segments...
May 13 08:23:44.604157373 (301h35m55.124981108s) [noise=true , address=145.79.227.50:4374   , piece_id=VU7FGWLJIQ6BJWLWR4V3FZ2OOKWZ4XUXXIK6UTIOXG4JQRS5QAVQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.616951337 (301h35m55.137775082s) [noise=true , address=145.79.227.45:4233   , piece_id=25YQOKEQLNYGD2DZBPWQD5HMUKBXEA2YMYVLRZWQVHCPXWKFOA3Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.626138077 (301h35m55.146961822s) [noise=true , address=145.79.227.7:4297    , piece_id=W2ZWVO5KC5ZJMC5LZKQ3ZBGEOBGE2Z7DN6BLRZCGPJ2LWDFEIAGA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627298322 (301h35m55.148122067s) [noise=true , address=145.79.227.3:4152    , piece_id=BZGI2SX3KLTQ5L74GQTXWJ3MXN5KLLFZKMPES5FRGGCLUD7XXV3A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627592093 (301h35m55.148415838s) [noise=true , address=145.79.227.52:4447   , piece_id=XTJJIUVCL2WH5TJ4CZRJAUETVQIYYB55KALFX46YG2FPOMLLOPUA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627893708 (301h35m55.148717453s) [noise=true , address=145.79.227.6:4263    , piece_id=MPF6TBBYJ3IQVVBMS36BIV6PAITZ25AYM3TJAI2KO7XBZB2FN7WQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.630105298 (301h35m55.150929043s) [noise=true , address=145.79.227.39:4017   , piece_id=H2XFKXRSXEAG4WM7XDSYFLSHUOECVFXR47PBYVHW4DZLKJXWLU5A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.635153522 (301h35m55.155977267s) [noise=true , address=145.79.227.25:4619   , piece_id=3EFGZMPN4XP4F4U5OKK5VIYD44FT3WXQB73NZQL4LFYVQMTSUIVA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.639359247 (301h35m55.160182992s) [noise=true , address=145.79.227.4:4185    , piece_id=73P5QCJDK5IESMP53DQNMZHRSOHFDVYXEM6AWW7CWR7PDDKH4EXQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.642768653 (301h35m55.163592388s) [noise=true , address=145.79.227.31:4798   , piece_id=2ZEX5FUWBC2NYJJCGZ5UMPBGKKXNGXFH27CMWB7I55BWAQ4X33XQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.669518538 (301h35m55.190342283s) [noise=true , address=145.79.227.13:4804   , piece_id=GAB5UUUHLI3P6YSFVQ6HHBTPCTLQOPE6THKSHKZPSAHLI7G2QFHA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.673346913 (301h35m55.194170658s) [noise=true , address=145.79.227.26:4654   , piece_id=OYHDYSBP5LMUP6ZQONZNELXVOY53ZSJMG33ONRQLLFGYPFKR3P5Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.722674662 (301h35m55.243498407s) [noise=true , address=77.90.31.15:4701     , piece_id=XQI7GRPTXPE35WKWKEQX72RSEQ74NMF7645CXJA2E5FKWWKHEI2A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.722697015 (301h35m55.24352076s) [seg_pos={0 0}, upload=288931]: Satisfied cohort requirements after 13 pieces
May 13 08:23:44.722987831 (301h35m55.243811566s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
May 13 08:23:44.722997429 (301h35m55.243821174s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723038947 (301h35m55.243862692s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Kwa9zxc9LDp78psKPNV55aP5RYvtcQi1NjphTkXBWRugwhg7s): protocol: write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection; write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection
May 13 08:23:44.723054898 (301h35m55.243878643s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723109372 (301h35m55.243933117s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XeA4A8wbmv7wgPjRQnNbP9xgDQz4MzJH7PasaYsTrsMzTJRhC): protocol: expected piece hash; context canceled; EOF
May 13 08:23:44.723116526 (301h35m55.243940281s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723129540 (301h35m55.243953285s) [seg_pos={0 0}, upload=288931]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 13 08:23:44.723189946 (301h35m55.244013691s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 13 08:23:44.723193784 (301h35m55.244017519s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 13 08:24:49.413522848 (301h36m59.934346603s) [upload=288931]: Done uploading segments...
May 13 08:24:49.413641645 (301h36m59.93446539s) [upload=288931]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.687192146s core uplink us-select-1
checkers.HTTP{"GET" "http://timesync.objectmount.io/health", ExpectedStatus: 200} failure (antiblip window 3/60): (http.timesync_objectmount_io_health)
antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................]
duration:
432.878125ms
target:
http://timesync.objectmount.io/health
trace:
https://jaeger.dev.storj.io/trace/498a8e7089f2c4a8
FAIL (2026-05-13 08:39:04 UTC)
orbiter warning: Get "http://timesync.objectmount.io/health": dial tcp 35.209.219.1:80: i/o timeout
Duration: 432.905378ms object-mount
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)
antiblip:
2 out of 60 is failed (threshold=3) [..................................X...X.....................]
duration:
1.901881921s
trace:
https://jaeger.dev.storj.io/trace/6b68594ae31476df
FAIL (2026-05-13 08:38:59 UTC)
orbiter warning: 
operation: StatBucket
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
Duration: 1.901907389s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............]
duration:
659.389539ms
remote:
"tcp:34.4.32.59:443"
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/68a242383b75c35d
FAIL (2026-05-13 08:38:59 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 659.414756ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................]
duration:
1.967368617s
trace:
https://jaeger.dev.storj.io/trace/3821857227b01f0d
FAIL (2026-05-13 08:38:49 UTC)
orbiter warning: operation: StatBucket
uplink: bucket: metaclient: context deadline exceeded
Duration: 1.967395047s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)
antiblip:
2 out of 60 is failed (threshold=3) [...................................X...X....................]
duration:
1.870055784s
trace:
https://jaeger.dev.storj.io/trace/6b68594ae31476df
FAIL (2026-05-13 08:38:28 UTC)
orbiter warning: 
operation: StatBucket
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
Duration: 1.870082676s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................]
duration:
1.975300647s
trace:
https://jaeger.dev.storj.io/trace/3821857227b01f0d
FAIL (2026-05-13 08:38:27 UTC)
orbiter warning: operation: StatBucket
uplink: bucket: metaclient: context deadline exceeded
Duration: 1.975330765s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................]
duration:
5.624399419s
trace:
https://jaeger.dev.storj.io/trace/f253484fd40b3cf
FAIL (2026-05-13 08:38:22 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
52.886159919s) [upload=288931]: Starting upload
May 13 08:23:42.365411428 (301h35m52.886235173s) [upload=288931]: Done starting upload
May 13 08:23:42.365500037 (301h35m52.886323782s) [upload=288931]: Uploading segments...
May 13 08:23:42.365647579 (301h35m52.886471334s) [upload=288931]: Got next segment. Inline: false
May 13 08:23:42.365683176 (301h35m52.886506911s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 13 08:23:42.365687885 (301h35m52.88651163s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 13 08:23:42.598383896 (301h35m53.119207641s) [seg_pos={0 0}, upload=288931]: Begin upload segment...
May 13 08:23:42.598464609 (301h35m53.119288344s) [seg_pos={0 0}, upload=288931]: Done begin upload segment.
May 13 08:23:42.598489827 (301h35m53.119313562s) [noise=true , address=145.79.227.4:4185    , piece_id=73P5QCJDK5IESMP53DQNMZHRSOHFDVYXEM6AWW7CWR7PDDKH4EXQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.598584227 (301h35m53.119407972s) [noise=true , address=77.90.31.15:4701     , piece_id=XQI7GRPTXPE35WKWKEQX72RSEQ74NMF7645CXJA2E5FKWWKHEI2A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599099290 (301h35m53.119923035s) [noise=true , address=145.79.227.52:4447   , piece_id=XTJJIUVCL2WH5TJ4CZRJAUETVQIYYB55KALFX46YG2FPOMLLOPUA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599448427 (301h35m53.120272172s) [noise=true , address=145.79.227.3:4152    , piece_id=BZGI2SX3KLTQ5L74GQTXWJ3MXN5KLLFZKMPES5FRGGCLUD7XXV3A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599785570 (301h35m53.120609315s) [noise=true , address=145.79.227.7:4297    , piece_id=W2ZWVO5KC5ZJMC5LZKQ3ZBGEOBGE2Z7DN6BLRZCGPJ2LWDFEIAGA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600089149 (301h35m53.120912894s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600393219 (301h35m53.121216964s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600679686 (301h35m53.121503431s) [noise=true , address=145.79.227.6:4263    , piece_id=MPF6TBBYJ3IQVVBMS36BIV6PAITZ25AYM3TJAI2KO7XBZB2FN7WQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.601010748 (301h35m53.121834493s) [noise=true , address=145.79.227.26:4654   , piece_id=OYHDYSBP5LMUP6ZQONZNELXVOY53ZSJMG33ONRQLLFGYPFKR3P5Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.601837916 (301h35m53.122661651s) [noise=true , address=145.79.227.39:4017   , piece_id=H2XFKXRSXEAG4WM7XDSYFLSHUOECVFXR47PBYVHW4DZLKJXWLU5A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.602900945 (301h35m53.12372469s) [noise=true , address=145.79.227.45:4233   , piece_id=25YQOKEQLNYGD2DZBPWQD5HMUKBXEA2YMYVLRZWQVHCPXWKFOA3Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.603751337 (301h35m53.124575072s) [noise=true , address=145.79.227.25:4619   , piece_id=3EFGZMPN4XP4F4U5OKK5VIYD44FT3WXQB73NZQL4LFYVQMTSUIVA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.604687004 (301h35m53.125510749s) [noise=true , address=145.79.227.13:4804   , piece_id=GAB5UUUHLI3P6YSFVQ6HHBTPCTLQOPE6THKSHKZPSAHLI7G2QFHA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.605665119 (301h35m53.126488864s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.606795627 (301h35m53.127619372s) [noise=true , address=145.79.227.50:4374   , piece_id=VU7FGWLJIQ6BJWLWR4V3FZ2OOKWZ4XUXXIK6UTIOXG4JQRS5QAVQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.607929281 (301h35m53.128753026s) [noise=true , address=145.79.227.31:4798   , piece_id=2ZEX5FUWBC2NYJJCGZ5UMPBGKKXNGXFH27CMWB7I55BWAQ4X33XQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:44.318577649 (301h35m54.839401394s) [upload=288931]: Next returned nil segment
May 13 08:23:44.318598147 (301h35m54.839421892s) [upload=288931]: Waiting for error group managing segments...
May 13 08:23:44.604157373 (301h35m55.124981108s) [noise=true , address=145.79.227.50:4374   , piece_id=VU7FGWLJIQ6BJWLWR4V3FZ2OOKWZ4XUXXIK6UTIOXG4JQRS5QAVQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.616951337 (301h35m55.137775082s) [noise=true , address=145.79.227.45:4233   , piece_id=25YQOKEQLNYGD2DZBPWQD5HMUKBXEA2YMYVLRZWQVHCPXWKFOA3Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.626138077 (301h35m55.146961822s) [noise=true , address=145.79.227.7:4297    , piece_id=W2ZWVO5KC5ZJMC5LZKQ3ZBGEOBGE2Z7DN6BLRZCGPJ2LWDFEIAGA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627298322 (301h35m55.148122067s) [noise=true , address=145.79.227.3:4152    , piece_id=BZGI2SX3KLTQ5L74GQTXWJ3MXN5KLLFZKMPES5FRGGCLUD7XXV3A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627592093 (301h35m55.148415838s) [noise=true , address=145.79.227.52:4447   , piece_id=XTJJIUVCL2WH5TJ4CZRJAUETVQIYYB55KALFX46YG2FPOMLLOPUA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627893708 (301h35m55.148717453s) [noise=true , address=145.79.227.6:4263    , piece_id=MPF6TBBYJ3IQVVBMS36BIV6PAITZ25AYM3TJAI2KO7XBZB2FN7WQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.630105298 (301h35m55.150929043s) [noise=true , address=145.79.227.39:4017   , piece_id=H2XFKXRSXEAG4WM7XDSYFLSHUOECVFXR47PBYVHW4DZLKJXWLU5A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.635153522 (301h35m55.155977267s) [noise=true , address=145.79.227.25:4619   , piece_id=3EFGZMPN4XP4F4U5OKK5VIYD44FT3WXQB73NZQL4LFYVQMTSUIVA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.639359247 (301h35m55.160182992s) [noise=true , address=145.79.227.4:4185    , piece_id=73P5QCJDK5IESMP53DQNMZHRSOHFDVYXEM6AWW7CWR7PDDKH4EXQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.642768653 (301h35m55.163592388s) [noise=true , address=145.79.227.31:4798   , piece_id=2ZEX5FUWBC2NYJJCGZ5UMPBGKKXNGXFH27CMWB7I55BWAQ4X33XQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.669518538 (301h35m55.190342283s) [noise=true , address=145.79.227.13:4804   , piece_id=GAB5UUUHLI3P6YSFVQ6HHBTPCTLQOPE6THKSHKZPSAHLI7G2QFHA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.673346913 (301h35m55.194170658s) [noise=true , address=145.79.227.26:4654   , piece_id=OYHDYSBP5LMUP6ZQONZNELXVOY53ZSJMG33ONRQLLFGYPFKR3P5Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.722674662 (301h35m55.243498407s) [noise=true , address=77.90.31.15:4701     , piece_id=XQI7GRPTXPE35WKWKEQX72RSEQ74NMF7645CXJA2E5FKWWKHEI2A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.722697015 (301h35m55.24352076s) [seg_pos={0 0}, upload=288931]: Satisfied cohort requirements after 13 pieces
May 13 08:23:44.722987831 (301h35m55.243811566s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
May 13 08:23:44.722997429 (301h35m55.243821174s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723038947 (301h35m55.243862692s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Kwa9zxc9LDp78psKPNV55aP5RYvtcQi1NjphTkXBWRugwhg7s): protocol: write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection; write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection
May 13 08:23:44.723054898 (301h35m55.243878643s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723109372 (301h35m55.243933117s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XeA4A8wbmv7wgPjRQnNbP9xgDQz4MzJH7PasaYsTrsMzTJRhC): protocol: expected piece hash; context canceled; EOF
May 13 08:23:44.723116526 (301h35m55.243940281s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723129540 (301h35m55.243953285s) [seg_pos={0 0}, upload=288931]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 13 08:23:44.723189946 (301h35m55.244013691s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 13 08:23:44.723193784 (301h35m55.244017519s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 13 08:24:49.413522848 (301h36m59.934346603s) [upload=288931]: Done uploading segments...
May 13 08:24:49.413641645 (301h36m59.93446539s) [upload=288931]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.624461697s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........]
duration:
657.902147ms
remote:
"tcp:34.4.32.59:443"
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/68a242383b75c35d
FAIL (2026-05-13 08:37:51 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 657.926023ms core satellite us1
checkers.HTTP{"GET" "http://timesync.objectmount.io/health", ExpectedStatus: 200} failure (antiblip window 3/60): (http.timesync_objectmount_io_health)
antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................]
duration:
431.538039ms
target:
http://timesync.objectmount.io/health
trace:
https://jaeger.dev.storj.io/trace/498a8e7089f2c4a8
FAIL (2026-05-13 08:37:50 UTC)
orbiter warning: Get "http://timesync.objectmount.io/health": dial tcp 35.209.219.1:80: i/o timeout
Duration: 431.558918ms object-mount
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................]
duration:
1.881618009s
trace:
https://jaeger.dev.storj.io/trace/3821857227b01f0d
FAIL (2026-05-13 08:37:48 UTC)
orbiter warning: operation: StatBucket
uplink: bucket: metaclient: context deadline exceeded
Duration: 1.881641143s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................]
duration:
5.682411854s
trace:
https://jaeger.dev.storj.io/trace/f253484fd40b3cf
FAIL (2026-05-13 08:37:38 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
52.886159919s) [upload=288931]: Starting upload
May 13 08:23:42.365411428 (301h35m52.886235173s) [upload=288931]: Done starting upload
May 13 08:23:42.365500037 (301h35m52.886323782s) [upload=288931]: Uploading segments...
May 13 08:23:42.365647579 (301h35m52.886471334s) [upload=288931]: Got next segment. Inline: false
May 13 08:23:42.365683176 (301h35m52.886506911s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 13 08:23:42.365687885 (301h35m52.88651163s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 13 08:23:42.598383896 (301h35m53.119207641s) [seg_pos={0 0}, upload=288931]: Begin upload segment...
May 13 08:23:42.598464609 (301h35m53.119288344s) [seg_pos={0 0}, upload=288931]: Done begin upload segment.
May 13 08:23:42.598489827 (301h35m53.119313562s) [noise=true , address=145.79.227.4:4185    , piece_id=73P5QCJDK5IESMP53DQNMZHRSOHFDVYXEM6AWW7CWR7PDDKH4EXQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.598584227 (301h35m53.119407972s) [noise=true , address=77.90.31.15:4701     , piece_id=XQI7GRPTXPE35WKWKEQX72RSEQ74NMF7645CXJA2E5FKWWKHEI2A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599099290 (301h35m53.119923035s) [noise=true , address=145.79.227.52:4447   , piece_id=XTJJIUVCL2WH5TJ4CZRJAUETVQIYYB55KALFX46YG2FPOMLLOPUA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599448427 (301h35m53.120272172s) [noise=true , address=145.79.227.3:4152    , piece_id=BZGI2SX3KLTQ5L74GQTXWJ3MXN5KLLFZKMPES5FRGGCLUD7XXV3A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599785570 (301h35m53.120609315s) [noise=true , address=145.79.227.7:4297    , piece_id=W2ZWVO5KC5ZJMC5LZKQ3ZBGEOBGE2Z7DN6BLRZCGPJ2LWDFEIAGA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600089149 (301h35m53.120912894s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600393219 (301h35m53.121216964s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600679686 (301h35m53.121503431s) [noise=true , address=145.79.227.6:4263    , piece_id=MPF6TBBYJ3IQVVBMS36BIV6PAITZ25AYM3TJAI2KO7XBZB2FN7WQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.601010748 (301h35m53.121834493s) [noise=true , address=145.79.227.26:4654   , piece_id=OYHDYSBP5LMUP6ZQONZNELXVOY53ZSJMG33ONRQLLFGYPFKR3P5Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.601837916 (301h35m53.122661651s) [noise=true , address=145.79.227.39:4017   , piece_id=H2XFKXRSXEAG4WM7XDSYFLSHUOECVFXR47PBYVHW4DZLKJXWLU5A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.602900945 (301h35m53.12372469s) [noise=true , address=145.79.227.45:4233   , piece_id=25YQOKEQLNYGD2DZBPWQD5HMUKBXEA2YMYVLRZWQVHCPXWKFOA3Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.603751337 (301h35m53.124575072s) [noise=true , address=145.79.227.25:4619   , piece_id=3EFGZMPN4XP4F4U5OKK5VIYD44FT3WXQB73NZQL4LFYVQMTSUIVA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.604687004 (301h35m53.125510749s) [noise=true , address=145.79.227.13:4804   , piece_id=GAB5UUUHLI3P6YSFVQ6HHBTPCTLQOPE6THKSHKZPSAHLI7G2QFHA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.605665119 (301h35m53.126488864s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.606795627 (301h35m53.127619372s) [noise=true , address=145.79.227.50:4374   , piece_id=VU7FGWLJIQ6BJWLWR4V3FZ2OOKWZ4XUXXIK6UTIOXG4JQRS5QAVQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.607929281 (301h35m53.128753026s) [noise=true , address=145.79.227.31:4798   , piece_id=2ZEX5FUWBC2NYJJCGZ5UMPBGKKXNGXFH27CMWB7I55BWAQ4X33XQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:44.318577649 (301h35m54.839401394s) [upload=288931]: Next returned nil segment
May 13 08:23:44.318598147 (301h35m54.839421892s) [upload=288931]: Waiting for error group managing segments...
May 13 08:23:44.604157373 (301h35m55.124981108s) [noise=true , address=145.79.227.50:4374   , piece_id=VU7FGWLJIQ6BJWLWR4V3FZ2OOKWZ4XUXXIK6UTIOXG4JQRS5QAVQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.616951337 (301h35m55.137775082s) [noise=true , address=145.79.227.45:4233   , piece_id=25YQOKEQLNYGD2DZBPWQD5HMUKBXEA2YMYVLRZWQVHCPXWKFOA3Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.626138077 (301h35m55.146961822s) [noise=true , address=145.79.227.7:4297    , piece_id=W2ZWVO5KC5ZJMC5LZKQ3ZBGEOBGE2Z7DN6BLRZCGPJ2LWDFEIAGA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627298322 (301h35m55.148122067s) [noise=true , address=145.79.227.3:4152    , piece_id=BZGI2SX3KLTQ5L74GQTXWJ3MXN5KLLFZKMPES5FRGGCLUD7XXV3A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627592093 (301h35m55.148415838s) [noise=true , address=145.79.227.52:4447   , piece_id=XTJJIUVCL2WH5TJ4CZRJAUETVQIYYB55KALFX46YG2FPOMLLOPUA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627893708 (301h35m55.148717453s) [noise=true , address=145.79.227.6:4263    , piece_id=MPF6TBBYJ3IQVVBMS36BIV6PAITZ25AYM3TJAI2KO7XBZB2FN7WQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.630105298 (301h35m55.150929043s) [noise=true , address=145.79.227.39:4017   , piece_id=H2XFKXRSXEAG4WM7XDSYFLSHUOECVFXR47PBYVHW4DZLKJXWLU5A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.635153522 (301h35m55.155977267s) [noise=true , address=145.79.227.25:4619   , piece_id=3EFGZMPN4XP4F4U5OKK5VIYD44FT3WXQB73NZQL4LFYVQMTSUIVA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.639359247 (301h35m55.160182992s) [noise=true , address=145.79.227.4:4185    , piece_id=73P5QCJDK5IESMP53DQNMZHRSOHFDVYXEM6AWW7CWR7PDDKH4EXQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.642768653 (301h35m55.163592388s) [noise=true , address=145.79.227.31:4798   , piece_id=2ZEX5FUWBC2NYJJCGZ5UMPBGKKXNGXFH27CMWB7I55BWAQ4X33XQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.669518538 (301h35m55.190342283s) [noise=true , address=145.79.227.13:4804   , piece_id=GAB5UUUHLI3P6YSFVQ6HHBTPCTLQOPE6THKSHKZPSAHLI7G2QFHA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.673346913 (301h35m55.194170658s) [noise=true , address=145.79.227.26:4654   , piece_id=OYHDYSBP5LMUP6ZQONZNELXVOY53ZSJMG33ONRQLLFGYPFKR3P5Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.722674662 (301h35m55.243498407s) [noise=true , address=77.90.31.15:4701     , piece_id=XQI7GRPTXPE35WKWKEQX72RSEQ74NMF7645CXJA2E5FKWWKHEI2A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.722697015 (301h35m55.24352076s) [seg_pos={0 0}, upload=288931]: Satisfied cohort requirements after 13 pieces
May 13 08:23:44.722987831 (301h35m55.243811566s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
May 13 08:23:44.722997429 (301h35m55.243821174s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723038947 (301h35m55.243862692s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Kwa9zxc9LDp78psKPNV55aP5RYvtcQi1NjphTkXBWRugwhg7s): protocol: write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection; write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection
May 13 08:23:44.723054898 (301h35m55.243878643s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723109372 (301h35m55.243933117s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XeA4A8wbmv7wgPjRQnNbP9xgDQz4MzJH7PasaYsTrsMzTJRhC): protocol: expected piece hash; context canceled; EOF
May 13 08:23:44.723116526 (301h35m55.243940281s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723129540 (301h35m55.243953285s) [seg_pos={0 0}, upload=288931]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 13 08:23:44.723189946 (301h35m55.244013691s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 13 08:23:44.723193784 (301h35m55.244017519s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 13 08:24:49.413522848 (301h36m59.934346603s) [upload=288931]: Done uploading segments...
May 13 08:24:49.413641645 (301h36m59.93446539s) [upload=288931]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.682437542s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)
antiblip:
2 out of 60 is failed (threshold=3) [....................................X...X...................]
duration:
1.875915406s
trace:
https://jaeger.dev.storj.io/trace/6b68594ae31476df
FAIL (2026-05-13 08:37:26 UTC)
orbiter warning: 
operation: StatBucket
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
Duration: 1.875940104s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)
antiblip:
2 out of 60 is failed (threshold=3) [.....................................X...X..................]
duration:
1.876379813s
trace:
https://jaeger.dev.storj.io/trace/6b68594ae31476df
FAIL (2026-05-13 08:37:05 UTC)
orbiter warning: 
operation: StatBucket
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
Duration: 1.87640434s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................]
duration:
5.623478775s
trace:
https://jaeger.dev.storj.io/trace/f253484fd40b3cf
FAIL (2026-05-13 08:37:02 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
52.886159919s) [upload=288931]: Starting upload
May 13 08:23:42.365411428 (301h35m52.886235173s) [upload=288931]: Done starting upload
May 13 08:23:42.365500037 (301h35m52.886323782s) [upload=288931]: Uploading segments...
May 13 08:23:42.365647579 (301h35m52.886471334s) [upload=288931]: Got next segment. Inline: false
May 13 08:23:42.365683176 (301h35m52.886506911s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 13 08:23:42.365687885 (301h35m52.88651163s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 13 08:23:42.598383896 (301h35m53.119207641s) [seg_pos={0 0}, upload=288931]: Begin upload segment...
May 13 08:23:42.598464609 (301h35m53.119288344s) [seg_pos={0 0}, upload=288931]: Done begin upload segment.
May 13 08:23:42.598489827 (301h35m53.119313562s) [noise=true , address=145.79.227.4:4185    , piece_id=73P5QCJDK5IESMP53DQNMZHRSOHFDVYXEM6AWW7CWR7PDDKH4EXQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.598584227 (301h35m53.119407972s) [noise=true , address=77.90.31.15:4701     , piece_id=XQI7GRPTXPE35WKWKEQX72RSEQ74NMF7645CXJA2E5FKWWKHEI2A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599099290 (301h35m53.119923035s) [noise=true , address=145.79.227.52:4447   , piece_id=XTJJIUVCL2WH5TJ4CZRJAUETVQIYYB55KALFX46YG2FPOMLLOPUA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599448427 (301h35m53.120272172s) [noise=true , address=145.79.227.3:4152    , piece_id=BZGI2SX3KLTQ5L74GQTXWJ3MXN5KLLFZKMPES5FRGGCLUD7XXV3A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599785570 (301h35m53.120609315s) [noise=true , address=145.79.227.7:4297    , piece_id=W2ZWVO5KC5ZJMC5LZKQ3ZBGEOBGE2Z7DN6BLRZCGPJ2LWDFEIAGA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600089149 (301h35m53.120912894s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600393219 (301h35m53.121216964s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600679686 (301h35m53.121503431s) [noise=true , address=145.79.227.6:4263    , piece_id=MPF6TBBYJ3IQVVBMS36BIV6PAITZ25AYM3TJAI2KO7XBZB2FN7WQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.601010748 (301h35m53.121834493s) [noise=true , address=145.79.227.26:4654   , piece_id=OYHDYSBP5LMUP6ZQONZNELXVOY53ZSJMG33ONRQLLFGYPFKR3P5Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.601837916 (301h35m53.122661651s) [noise=true , address=145.79.227.39:4017   , piece_id=H2XFKXRSXEAG4WM7XDSYFLSHUOECVFXR47PBYVHW4DZLKJXWLU5A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.602900945 (301h35m53.12372469s) [noise=true , address=145.79.227.45:4233   , piece_id=25YQOKEQLNYGD2DZBPWQD5HMUKBXEA2YMYVLRZWQVHCPXWKFOA3Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.603751337 (301h35m53.124575072s) [noise=true , address=145.79.227.25:4619   , piece_id=3EFGZMPN4XP4F4U5OKK5VIYD44FT3WXQB73NZQL4LFYVQMTSUIVA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.604687004 (301h35m53.125510749s) [noise=true , address=145.79.227.13:4804   , piece_id=GAB5UUUHLI3P6YSFVQ6HHBTPCTLQOPE6THKSHKZPSAHLI7G2QFHA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.605665119 (301h35m53.126488864s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.606795627 (301h35m53.127619372s) [noise=true , address=145.79.227.50:4374   , piece_id=VU7FGWLJIQ6BJWLWR4V3FZ2OOKWZ4XUXXIK6UTIOXG4JQRS5QAVQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.607929281 (301h35m53.128753026s) [noise=true , address=145.79.227.31:4798   , piece_id=2ZEX5FUWBC2NYJJCGZ5UMPBGKKXNGXFH27CMWB7I55BWAQ4X33XQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:44.318577649 (301h35m54.839401394s) [upload=288931]: Next returned nil segment
May 13 08:23:44.318598147 (301h35m54.839421892s) [upload=288931]: Waiting for error group managing segments...
May 13 08:23:44.604157373 (301h35m55.124981108s) [noise=true , address=145.79.227.50:4374   , piece_id=VU7FGWLJIQ6BJWLWR4V3FZ2OOKWZ4XUXXIK6UTIOXG4JQRS5QAVQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.616951337 (301h35m55.137775082s) [noise=true , address=145.79.227.45:4233   , piece_id=25YQOKEQLNYGD2DZBPWQD5HMUKBXEA2YMYVLRZWQVHCPXWKFOA3Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.626138077 (301h35m55.146961822s) [noise=true , address=145.79.227.7:4297    , piece_id=W2ZWVO5KC5ZJMC5LZKQ3ZBGEOBGE2Z7DN6BLRZCGPJ2LWDFEIAGA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627298322 (301h35m55.148122067s) [noise=true , address=145.79.227.3:4152    , piece_id=BZGI2SX3KLTQ5L74GQTXWJ3MXN5KLLFZKMPES5FRGGCLUD7XXV3A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627592093 (301h35m55.148415838s) [noise=true , address=145.79.227.52:4447   , piece_id=XTJJIUVCL2WH5TJ4CZRJAUETVQIYYB55KALFX46YG2FPOMLLOPUA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627893708 (301h35m55.148717453s) [noise=true , address=145.79.227.6:4263    , piece_id=MPF6TBBYJ3IQVVBMS36BIV6PAITZ25AYM3TJAI2KO7XBZB2FN7WQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.630105298 (301h35m55.150929043s) [noise=true , address=145.79.227.39:4017   , piece_id=H2XFKXRSXEAG4WM7XDSYFLSHUOECVFXR47PBYVHW4DZLKJXWLU5A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.635153522 (301h35m55.155977267s) [noise=true , address=145.79.227.25:4619   , piece_id=3EFGZMPN4XP4F4U5OKK5VIYD44FT3WXQB73NZQL4LFYVQMTSUIVA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.639359247 (301h35m55.160182992s) [noise=true , address=145.79.227.4:4185    , piece_id=73P5QCJDK5IESMP53DQNMZHRSOHFDVYXEM6AWW7CWR7PDDKH4EXQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.642768653 (301h35m55.163592388s) [noise=true , address=145.79.227.31:4798   , piece_id=2ZEX5FUWBC2NYJJCGZ5UMPBGKKXNGXFH27CMWB7I55BWAQ4X33XQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.669518538 (301h35m55.190342283s) [noise=true , address=145.79.227.13:4804   , piece_id=GAB5UUUHLI3P6YSFVQ6HHBTPCTLQOPE6THKSHKZPSAHLI7G2QFHA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.673346913 (301h35m55.194170658s) [noise=true , address=145.79.227.26:4654   , piece_id=OYHDYSBP5LMUP6ZQONZNELXVOY53ZSJMG33ONRQLLFGYPFKR3P5Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.722674662 (301h35m55.243498407s) [noise=true , address=77.90.31.15:4701     , piece_id=XQI7GRPTXPE35WKWKEQX72RSEQ74NMF7645CXJA2E5FKWWKHEI2A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.722697015 (301h35m55.24352076s) [seg_pos={0 0}, upload=288931]: Satisfied cohort requirements after 13 pieces
May 13 08:23:44.722987831 (301h35m55.243811566s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
May 13 08:23:44.722997429 (301h35m55.243821174s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723038947 (301h35m55.243862692s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Kwa9zxc9LDp78psKPNV55aP5RYvtcQi1NjphTkXBWRugwhg7s): protocol: write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection; write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection
May 13 08:23:44.723054898 (301h35m55.243878643s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723109372 (301h35m55.243933117s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XeA4A8wbmv7wgPjRQnNbP9xgDQz4MzJH7PasaYsTrsMzTJRhC): protocol: expected piece hash; context canceled; EOF
May 13 08:23:44.723116526 (301h35m55.243940281s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723129540 (301h35m55.243953285s) [seg_pos={0 0}, upload=288931]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 13 08:23:44.723189946 (301h35m55.244013691s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 13 08:23:44.723193784 (301h35m55.244017519s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 13 08:24:49.413522848 (301h36m59.934346603s) [upload=288931]: Done uploading segments...
May 13 08:24:49.413641645 (301h36m59.93446539s) [upload=288931]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.623503562s core uplink us-select-1
checkers.HTTP{"GET" "http://timesync.objectmount.io/health", ExpectedStatus: 200} failure (antiblip window 3/60): (http.timesync_objectmount_io_health)
antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................]
duration:
430.421336ms
target:
http://timesync.objectmount.io/health
trace:
https://jaeger.dev.storj.io/trace/498a8e7089f2c4a8
FAIL (2026-05-13 08:36:53 UTC)
orbiter warning: Get "http://timesync.objectmount.io/health": dial tcp 35.209.219.1:80: i/o timeout
Duration: 430.445062ms object-mount
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................]
duration:
1.883561818s
trace:
https://jaeger.dev.storj.io/trace/3821857227b01f0d
FAIL (2026-05-13 08:36:52 UTC)
orbiter warning: operation: StatBucket
uplink: bucket: metaclient: context deadline exceeded
Duration: 1.883585212s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)
antiblip:
2 out of 60 is failed (threshold=3) [......................................X...X.................]
duration:
1.885259157s
trace:
https://jaeger.dev.storj.io/trace/6b68594ae31476df
FAIL (2026-05-13 08:36:45 UTC)
orbiter warning: 
operation: StatBucket
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
Duration: 1.885286118s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........]
duration:
652.717734ms
remote:
"tcp:34.4.32.59:443"
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/68a242383b75c35d
FAIL (2026-05-13 08:36:39 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 652.754465ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................]
duration:
5.691128302s
trace:
https://jaeger.dev.storj.io/trace/f253484fd40b3cf
FAIL (2026-05-13 08:36:16 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
52.886159919s) [upload=288931]: Starting upload
May 13 08:23:42.365411428 (301h35m52.886235173s) [upload=288931]: Done starting upload
May 13 08:23:42.365500037 (301h35m52.886323782s) [upload=288931]: Uploading segments...
May 13 08:23:42.365647579 (301h35m52.886471334s) [upload=288931]: Got next segment. Inline: false
May 13 08:23:42.365683176 (301h35m52.886506911s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 13 08:23:42.365687885 (301h35m52.88651163s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 13 08:23:42.598383896 (301h35m53.119207641s) [seg_pos={0 0}, upload=288931]: Begin upload segment...
May 13 08:23:42.598464609 (301h35m53.119288344s) [seg_pos={0 0}, upload=288931]: Done begin upload segment.
May 13 08:23:42.598489827 (301h35m53.119313562s) [noise=true , address=145.79.227.4:4185    , piece_id=73P5QCJDK5IESMP53DQNMZHRSOHFDVYXEM6AWW7CWR7PDDKH4EXQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.598584227 (301h35m53.119407972s) [noise=true , address=77.90.31.15:4701     , piece_id=XQI7GRPTXPE35WKWKEQX72RSEQ74NMF7645CXJA2E5FKWWKHEI2A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599099290 (301h35m53.119923035s) [noise=true , address=145.79.227.52:4447   , piece_id=XTJJIUVCL2WH5TJ4CZRJAUETVQIYYB55KALFX46YG2FPOMLLOPUA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599448427 (301h35m53.120272172s) [noise=true , address=145.79.227.3:4152    , piece_id=BZGI2SX3KLTQ5L74GQTXWJ3MXN5KLLFZKMPES5FRGGCLUD7XXV3A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599785570 (301h35m53.120609315s) [noise=true , address=145.79.227.7:4297    , piece_id=W2ZWVO5KC5ZJMC5LZKQ3ZBGEOBGE2Z7DN6BLRZCGPJ2LWDFEIAGA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600089149 (301h35m53.120912894s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600393219 (301h35m53.121216964s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600679686 (301h35m53.121503431s) [noise=true , address=145.79.227.6:4263    , piece_id=MPF6TBBYJ3IQVVBMS36BIV6PAITZ25AYM3TJAI2KO7XBZB2FN7WQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.601010748 (301h35m53.121834493s) [noise=true , address=145.79.227.26:4654   , piece_id=OYHDYSBP5LMUP6ZQONZNELXVOY53ZSJMG33ONRQLLFGYPFKR3P5Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.601837916 (301h35m53.122661651s) [noise=true , address=145.79.227.39:4017   , piece_id=H2XFKXRSXEAG4WM7XDSYFLSHUOECVFXR47PBYVHW4DZLKJXWLU5A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.602900945 (301h35m53.12372469s) [noise=true , address=145.79.227.45:4233   , piece_id=25YQOKEQLNYGD2DZBPWQD5HMUKBXEA2YMYVLRZWQVHCPXWKFOA3Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.603751337 (301h35m53.124575072s) [noise=true , address=145.79.227.25:4619   , piece_id=3EFGZMPN4XP4F4U5OKK5VIYD44FT3WXQB73NZQL4LFYVQMTSUIVA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.604687004 (301h35m53.125510749s) [noise=true , address=145.79.227.13:4804   , piece_id=GAB5UUUHLI3P6YSFVQ6HHBTPCTLQOPE6THKSHKZPSAHLI7G2QFHA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.605665119 (301h35m53.126488864s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.606795627 (301h35m53.127619372s) [noise=true , address=145.79.227.50:4374   , piece_id=VU7FGWLJIQ6BJWLWR4V3FZ2OOKWZ4XUXXIK6UTIOXG4JQRS5QAVQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.607929281 (301h35m53.128753026s) [noise=true , address=145.79.227.31:4798   , piece_id=2ZEX5FUWBC2NYJJCGZ5UMPBGKKXNGXFH27CMWB7I55BWAQ4X33XQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:44.318577649 (301h35m54.839401394s) [upload=288931]: Next returned nil segment
May 13 08:23:44.318598147 (301h35m54.839421892s) [upload=288931]: Waiting for error group managing segments...
May 13 08:23:44.604157373 (301h35m55.124981108s) [noise=true , address=145.79.227.50:4374   , piece_id=VU7FGWLJIQ6BJWLWR4V3FZ2OOKWZ4XUXXIK6UTIOXG4JQRS5QAVQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.616951337 (301h35m55.137775082s) [noise=true , address=145.79.227.45:4233   , piece_id=25YQOKEQLNYGD2DZBPWQD5HMUKBXEA2YMYVLRZWQVHCPXWKFOA3Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.626138077 (301h35m55.146961822s) [noise=true , address=145.79.227.7:4297    , piece_id=W2ZWVO5KC5ZJMC5LZKQ3ZBGEOBGE2Z7DN6BLRZCGPJ2LWDFEIAGA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627298322 (301h35m55.148122067s) [noise=true , address=145.79.227.3:4152    , piece_id=BZGI2SX3KLTQ5L74GQTXWJ3MXN5KLLFZKMPES5FRGGCLUD7XXV3A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627592093 (301h35m55.148415838s) [noise=true , address=145.79.227.52:4447   , piece_id=XTJJIUVCL2WH5TJ4CZRJAUETVQIYYB55KALFX46YG2FPOMLLOPUA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627893708 (301h35m55.148717453s) [noise=true , address=145.79.227.6:4263    , piece_id=MPF6TBBYJ3IQVVBMS36BIV6PAITZ25AYM3TJAI2KO7XBZB2FN7WQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.630105298 (301h35m55.150929043s) [noise=true , address=145.79.227.39:4017   , piece_id=H2XFKXRSXEAG4WM7XDSYFLSHUOECVFXR47PBYVHW4DZLKJXWLU5A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.635153522 (301h35m55.155977267s) [noise=true , address=145.79.227.25:4619   , piece_id=3EFGZMPN4XP4F4U5OKK5VIYD44FT3WXQB73NZQL4LFYVQMTSUIVA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.639359247 (301h35m55.160182992s) [noise=true , address=145.79.227.4:4185    , piece_id=73P5QCJDK5IESMP53DQNMZHRSOHFDVYXEM6AWW7CWR7PDDKH4EXQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.642768653 (301h35m55.163592388s) [noise=true , address=145.79.227.31:4798   , piece_id=2ZEX5FUWBC2NYJJCGZ5UMPBGKKXNGXFH27CMWB7I55BWAQ4X33XQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.669518538 (301h35m55.190342283s) [noise=true , address=145.79.227.13:4804   , piece_id=GAB5UUUHLI3P6YSFVQ6HHBTPCTLQOPE6THKSHKZPSAHLI7G2QFHA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.673346913 (301h35m55.194170658s) [noise=true , address=145.79.227.26:4654   , piece_id=OYHDYSBP5LMUP6ZQONZNELXVOY53ZSJMG33ONRQLLFGYPFKR3P5Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.722674662 (301h35m55.243498407s) [noise=true , address=77.90.31.15:4701     , piece_id=XQI7GRPTXPE35WKWKEQX72RSEQ74NMF7645CXJA2E5FKWWKHEI2A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.722697015 (301h35m55.24352076s) [seg_pos={0 0}, upload=288931]: Satisfied cohort requirements after 13 pieces
May 13 08:23:44.722987831 (301h35m55.243811566s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
May 13 08:23:44.722997429 (301h35m55.243821174s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723038947 (301h35m55.243862692s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Kwa9zxc9LDp78psKPNV55aP5RYvtcQi1NjphTkXBWRugwhg7s): protocol: write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection; write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection
May 13 08:23:44.723054898 (301h35m55.243878643s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723109372 (301h35m55.243933117s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XeA4A8wbmv7wgPjRQnNbP9xgDQz4MzJH7PasaYsTrsMzTJRhC): protocol: expected piece hash; context canceled; EOF
May 13 08:23:44.723116526 (301h35m55.243940281s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723129540 (301h35m55.243953285s) [seg_pos={0 0}, upload=288931]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 13 08:23:44.723189946 (301h35m55.244013691s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 13 08:23:44.723193784 (301h35m55.244017519s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 13 08:24:49.413522848 (301h36m59.934346603s) [upload=288931]: Done uploading segments...
May 13 08:24:49.413641645 (301h36m59.93446539s) [upload=288931]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.691157719s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................]
duration:
1.876672832s
trace:
https://jaeger.dev.storj.io/trace/3821857227b01f0d
FAIL (2026-05-13 08:35:59 UTC)
orbiter warning: operation: StatBucket
uplink: bucket: metaclient: context deadline exceeded
Duration: 1.876691947s core uplink us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........]
duration:
651.706191ms
remote:
"tcp:34.4.32.59:443"
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/68a242383b75c35d
FAIL (2026-05-13 08:35:59 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 651.728654ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)
antiblip:
2 out of 60 is failed (threshold=3) [.......................................X...X................]
duration:
1.873518504s
trace:
https://jaeger.dev.storj.io/trace/6b68594ae31476df
FAIL (2026-05-13 08:35:41 UTC)
orbiter warning: 
operation: StatBucket
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
Duration: 1.873552468s core uplink us-select-1
checkers.HTTP{"GET" "http://timesync.objectmount.io/health", ExpectedStatus: 200} failure (antiblip window 3/60): (http.timesync_objectmount_io_health)
antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................]
duration:
430.728261ms
target:
http://timesync.objectmount.io/health
trace:
https://jaeger.dev.storj.io/trace/498a8e7089f2c4a8
FAIL (2026-05-13 08:35:40 UTC)
orbiter warning: Get "http://timesync.objectmount.io/health": dial tcp 35.209.219.1:80: i/o timeout
Duration: 430.757699ms object-mount
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)
antiblip:
2 out of 60 is failed (threshold=3) [........................................X...X...............]
duration:
1.912297183s
trace:
https://jaeger.dev.storj.io/trace/6b68594ae31476df
FAIL (2026-05-13 08:35:25 UTC)
orbiter warning: 
operation: StatBucket
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
Duration: 1.912330367s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............]
duration:
5.631642445s
trace:
https://jaeger.dev.storj.io/trace/f253484fd40b3cf
FAIL (2026-05-13 08:35:11 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
52.886159919s) [upload=288931]: Starting upload
May 13 08:23:42.365411428 (301h35m52.886235173s) [upload=288931]: Done starting upload
May 13 08:23:42.365500037 (301h35m52.886323782s) [upload=288931]: Uploading segments...
May 13 08:23:42.365647579 (301h35m52.886471334s) [upload=288931]: Got next segment. Inline: false
May 13 08:23:42.365683176 (301h35m52.886506911s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 13 08:23:42.365687885 (301h35m52.88651163s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 13 08:23:42.598383896 (301h35m53.119207641s) [seg_pos={0 0}, upload=288931]: Begin upload segment...
May 13 08:23:42.598464609 (301h35m53.119288344s) [seg_pos={0 0}, upload=288931]: Done begin upload segment.
May 13 08:23:42.598489827 (301h35m53.119313562s) [noise=true , address=145.79.227.4:4185    , piece_id=73P5QCJDK5IESMP53DQNMZHRSOHFDVYXEM6AWW7CWR7PDDKH4EXQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.598584227 (301h35m53.119407972s) [noise=true , address=77.90.31.15:4701     , piece_id=XQI7GRPTXPE35WKWKEQX72RSEQ74NMF7645CXJA2E5FKWWKHEI2A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599099290 (301h35m53.119923035s) [noise=true , address=145.79.227.52:4447   , piece_id=XTJJIUVCL2WH5TJ4CZRJAUETVQIYYB55KALFX46YG2FPOMLLOPUA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599448427 (301h35m53.120272172s) [noise=true , address=145.79.227.3:4152    , piece_id=BZGI2SX3KLTQ5L74GQTXWJ3MXN5KLLFZKMPES5FRGGCLUD7XXV3A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599785570 (301h35m53.120609315s) [noise=true , address=145.79.227.7:4297    , piece_id=W2ZWVO5KC5ZJMC5LZKQ3ZBGEOBGE2Z7DN6BLRZCGPJ2LWDFEIAGA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600089149 (301h35m53.120912894s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600393219 (301h35m53.121216964s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600679686 (301h35m53.121503431s) [noise=true , address=145.79.227.6:4263    , piece_id=MPF6TBBYJ3IQVVBMS36BIV6PAITZ25AYM3TJAI2KO7XBZB2FN7WQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.601010748 (301h35m53.121834493s) [noise=true , address=145.79.227.26:4654   , piece_id=OYHDYSBP5LMUP6ZQONZNELXVOY53ZSJMG33ONRQLLFGYPFKR3P5Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.601837916 (301h35m53.122661651s) [noise=true , address=145.79.227.39:4017   , piece_id=H2XFKXRSXEAG4WM7XDSYFLSHUOECVFXR47PBYVHW4DZLKJXWLU5A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.602900945 (301h35m53.12372469s) [noise=true , address=145.79.227.45:4233   , piece_id=25YQOKEQLNYGD2DZBPWQD5HMUKBXEA2YMYVLRZWQVHCPXWKFOA3Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.603751337 (301h35m53.124575072s) [noise=true , address=145.79.227.25:4619   , piece_id=3EFGZMPN4XP4F4U5OKK5VIYD44FT3WXQB73NZQL4LFYVQMTSUIVA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.604687004 (301h35m53.125510749s) [noise=true , address=145.79.227.13:4804   , piece_id=GAB5UUUHLI3P6YSFVQ6HHBTPCTLQOPE6THKSHKZPSAHLI7G2QFHA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.605665119 (301h35m53.126488864s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.606795627 (301h35m53.127619372s) [noise=true , address=145.79.227.50:4374   , piece_id=VU7FGWLJIQ6BJWLWR4V3FZ2OOKWZ4XUXXIK6UTIOXG4JQRS5QAVQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.607929281 (301h35m53.128753026s) [noise=true , address=145.79.227.31:4798   , piece_id=2ZEX5FUWBC2NYJJCGZ5UMPBGKKXNGXFH27CMWB7I55BWAQ4X33XQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:44.318577649 (301h35m54.839401394s) [upload=288931]: Next returned nil segment
May 13 08:23:44.318598147 (301h35m54.839421892s) [upload=288931]: Waiting for error group managing segments...
May 13 08:23:44.604157373 (301h35m55.124981108s) [noise=true , address=145.79.227.50:4374   , piece_id=VU7FGWLJIQ6BJWLWR4V3FZ2OOKWZ4XUXXIK6UTIOXG4JQRS5QAVQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.616951337 (301h35m55.137775082s) [noise=true , address=145.79.227.45:4233   , piece_id=25YQOKEQLNYGD2DZBPWQD5HMUKBXEA2YMYVLRZWQVHCPXWKFOA3Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.626138077 (301h35m55.146961822s) [noise=true , address=145.79.227.7:4297    , piece_id=W2ZWVO5KC5ZJMC5LZKQ3ZBGEOBGE2Z7DN6BLRZCGPJ2LWDFEIAGA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627298322 (301h35m55.148122067s) [noise=true , address=145.79.227.3:4152    , piece_id=BZGI2SX3KLTQ5L74GQTXWJ3MXN5KLLFZKMPES5FRGGCLUD7XXV3A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627592093 (301h35m55.148415838s) [noise=true , address=145.79.227.52:4447   , piece_id=XTJJIUVCL2WH5TJ4CZRJAUETVQIYYB55KALFX46YG2FPOMLLOPUA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627893708 (301h35m55.148717453s) [noise=true , address=145.79.227.6:4263    , piece_id=MPF6TBBYJ3IQVVBMS36BIV6PAITZ25AYM3TJAI2KO7XBZB2FN7WQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.630105298 (301h35m55.150929043s) [noise=true , address=145.79.227.39:4017   , piece_id=H2XFKXRSXEAG4WM7XDSYFLSHUOECVFXR47PBYVHW4DZLKJXWLU5A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.635153522 (301h35m55.155977267s) [noise=true , address=145.79.227.25:4619   , piece_id=3EFGZMPN4XP4F4U5OKK5VIYD44FT3WXQB73NZQL4LFYVQMTSUIVA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.639359247 (301h35m55.160182992s) [noise=true , address=145.79.227.4:4185    , piece_id=73P5QCJDK5IESMP53DQNMZHRSOHFDVYXEM6AWW7CWR7PDDKH4EXQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.642768653 (301h35m55.163592388s) [noise=true , address=145.79.227.31:4798   , piece_id=2ZEX5FUWBC2NYJJCGZ5UMPBGKKXNGXFH27CMWB7I55BWAQ4X33XQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.669518538 (301h35m55.190342283s) [noise=true , address=145.79.227.13:4804   , piece_id=GAB5UUUHLI3P6YSFVQ6HHBTPCTLQOPE6THKSHKZPSAHLI7G2QFHA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.673346913 (301h35m55.194170658s) [noise=true , address=145.79.227.26:4654   , piece_id=OYHDYSBP5LMUP6ZQONZNELXVOY53ZSJMG33ONRQLLFGYPFKR3P5Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.722674662 (301h35m55.243498407s) [noise=true , address=77.90.31.15:4701     , piece_id=XQI7GRPTXPE35WKWKEQX72RSEQ74NMF7645CXJA2E5FKWWKHEI2A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.722697015 (301h35m55.24352076s) [seg_pos={0 0}, upload=288931]: Satisfied cohort requirements after 13 pieces
May 13 08:23:44.722987831 (301h35m55.243811566s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
May 13 08:23:44.722997429 (301h35m55.243821174s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723038947 (301h35m55.243862692s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Kwa9zxc9LDp78psKPNV55aP5RYvtcQi1NjphTkXBWRugwhg7s): protocol: write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection; write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection
May 13 08:23:44.723054898 (301h35m55.243878643s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723109372 (301h35m55.243933117s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XeA4A8wbmv7wgPjRQnNbP9xgDQz4MzJH7PasaYsTrsMzTJRhC): protocol: expected piece hash; context canceled; EOF
May 13 08:23:44.723116526 (301h35m55.243940281s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723129540 (301h35m55.243953285s) [seg_pos={0 0}, upload=288931]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 13 08:23:44.723189946 (301h35m55.244013691s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 13 08:23:44.723193784 (301h35m55.244017519s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 13 08:24:49.413522848 (301h36m59.934346603s) [upload=288931]: Done uploading segments...
May 13 08:24:49.413641645 (301h36m59.93446539s) [upload=288931]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.631668896s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................]
duration:
1.911702349s
trace:
https://jaeger.dev.storj.io/trace/3821857227b01f0d
FAIL (2026-05-13 08:35:04 UTC)
orbiter warning: operation: StatBucket
uplink: bucket: metaclient: context deadline exceeded
Duration: 1.911726846s core uplink us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........]
duration:
652.504877ms
remote:
"tcp:34.4.32.59:443"
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/68a242383b75c35d
FAIL (2026-05-13 08:34:45 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 652.541637ms core satellite us1
checkers.HTTP{"GET" "http://timesync.objectmount.io/health", ExpectedStatus: 200} failure (antiblip window 3/60): (http.timesync_objectmount_io_health)
antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............]
duration:
431.759101ms
target:
http://timesync.objectmount.io/health
trace:
https://jaeger.dev.storj.io/trace/498a8e7089f2c4a8
FAIL (2026-05-13 08:34:42 UTC)
orbiter warning: Get "http://timesync.objectmount.io/health": dial tcp 35.209.219.1:80: i/o timeout
Duration: 431.81132ms object-mount
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)
antiblip:
2 out of 60 is failed (threshold=3) [.........................................X...X..............]
duration:
1.89384327s
trace:
https://jaeger.dev.storj.io/trace/6b68594ae31476df
FAIL (2026-05-13 08:34:41 UTC)
orbiter warning: 
operation: StatBucket
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
Duration: 1.893870171s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................]
duration:
1.858614059s
trace:
https://jaeger.dev.storj.io/trace/3821857227b01f0d
FAIL (2026-05-13 08:34:29 UTC)
orbiter warning: operation: StatBucket
uplink: bucket: metaclient: context deadline exceeded
Duration: 1.858642875s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............]
duration:
5.741738005s
trace:
https://jaeger.dev.storj.io/trace/f253484fd40b3cf
FAIL (2026-05-13 08:34:24 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
52.886159919s) [upload=288931]: Starting upload
May 13 08:23:42.365411428 (301h35m52.886235173s) [upload=288931]: Done starting upload
May 13 08:23:42.365500037 (301h35m52.886323782s) [upload=288931]: Uploading segments...
May 13 08:23:42.365647579 (301h35m52.886471334s) [upload=288931]: Got next segment. Inline: false
May 13 08:23:42.365683176 (301h35m52.886506911s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 13 08:23:42.365687885 (301h35m52.88651163s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 13 08:23:42.598383896 (301h35m53.119207641s) [seg_pos={0 0}, upload=288931]: Begin upload segment...
May 13 08:23:42.598464609 (301h35m53.119288344s) [seg_pos={0 0}, upload=288931]: Done begin upload segment.
May 13 08:23:42.598489827 (301h35m53.119313562s) [noise=true , address=145.79.227.4:4185    , piece_id=73P5QCJDK5IESMP53DQNMZHRSOHFDVYXEM6AWW7CWR7PDDKH4EXQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.598584227 (301h35m53.119407972s) [noise=true , address=77.90.31.15:4701     , piece_id=XQI7GRPTXPE35WKWKEQX72RSEQ74NMF7645CXJA2E5FKWWKHEI2A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599099290 (301h35m53.119923035s) [noise=true , address=145.79.227.52:4447   , piece_id=XTJJIUVCL2WH5TJ4CZRJAUETVQIYYB55KALFX46YG2FPOMLLOPUA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599448427 (301h35m53.120272172s) [noise=true , address=145.79.227.3:4152    , piece_id=BZGI2SX3KLTQ5L74GQTXWJ3MXN5KLLFZKMPES5FRGGCLUD7XXV3A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599785570 (301h35m53.120609315s) [noise=true , address=145.79.227.7:4297    , piece_id=W2ZWVO5KC5ZJMC5LZKQ3ZBGEOBGE2Z7DN6BLRZCGPJ2LWDFEIAGA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600089149 (301h35m53.120912894s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600393219 (301h35m53.121216964s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600679686 (301h35m53.121503431s) [noise=true , address=145.79.227.6:4263    , piece_id=MPF6TBBYJ3IQVVBMS36BIV6PAITZ25AYM3TJAI2KO7XBZB2FN7WQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.601010748 (301h35m53.121834493s) [noise=true , address=145.79.227.26:4654   , piece_id=OYHDYSBP5LMUP6ZQONZNELXVOY53ZSJMG33ONRQLLFGYPFKR3P5Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.601837916 (301h35m53.122661651s) [noise=true , address=145.79.227.39:4017   , piece_id=H2XFKXRSXEAG4WM7XDSYFLSHUOECVFXR47PBYVHW4DZLKJXWLU5A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.602900945 (301h35m53.12372469s) [noise=true , address=145.79.227.45:4233   , piece_id=25YQOKEQLNYGD2DZBPWQD5HMUKBXEA2YMYVLRZWQVHCPXWKFOA3Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.603751337 (301h35m53.124575072s) [noise=true , address=145.79.227.25:4619   , piece_id=3EFGZMPN4XP4F4U5OKK5VIYD44FT3WXQB73NZQL4LFYVQMTSUIVA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.604687004 (301h35m53.125510749s) [noise=true , address=145.79.227.13:4804   , piece_id=GAB5UUUHLI3P6YSFVQ6HHBTPCTLQOPE6THKSHKZPSAHLI7G2QFHA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.605665119 (301h35m53.126488864s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.606795627 (301h35m53.127619372s) [noise=true , address=145.79.227.50:4374   , piece_id=VU7FGWLJIQ6BJWLWR4V3FZ2OOKWZ4XUXXIK6UTIOXG4JQRS5QAVQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.607929281 (301h35m53.128753026s) [noise=true , address=145.79.227.31:4798   , piece_id=2ZEX5FUWBC2NYJJCGZ5UMPBGKKXNGXFH27CMWB7I55BWAQ4X33XQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:44.318577649 (301h35m54.839401394s) [upload=288931]: Next returned nil segment
May 13 08:23:44.318598147 (301h35m54.839421892s) [upload=288931]: Waiting for error group managing segments...
May 13 08:23:44.604157373 (301h35m55.124981108s) [noise=true , address=145.79.227.50:4374   , piece_id=VU7FGWLJIQ6BJWLWR4V3FZ2OOKWZ4XUXXIK6UTIOXG4JQRS5QAVQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.616951337 (301h35m55.137775082s) [noise=true , address=145.79.227.45:4233   , piece_id=25YQOKEQLNYGD2DZBPWQD5HMUKBXEA2YMYVLRZWQVHCPXWKFOA3Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.626138077 (301h35m55.146961822s) [noise=true , address=145.79.227.7:4297    , piece_id=W2ZWVO5KC5ZJMC5LZKQ3ZBGEOBGE2Z7DN6BLRZCGPJ2LWDFEIAGA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627298322 (301h35m55.148122067s) [noise=true , address=145.79.227.3:4152    , piece_id=BZGI2SX3KLTQ5L74GQTXWJ3MXN5KLLFZKMPES5FRGGCLUD7XXV3A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627592093 (301h35m55.148415838s) [noise=true , address=145.79.227.52:4447   , piece_id=XTJJIUVCL2WH5TJ4CZRJAUETVQIYYB55KALFX46YG2FPOMLLOPUA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627893708 (301h35m55.148717453s) [noise=true , address=145.79.227.6:4263    , piece_id=MPF6TBBYJ3IQVVBMS36BIV6PAITZ25AYM3TJAI2KO7XBZB2FN7WQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.630105298 (301h35m55.150929043s) [noise=true , address=145.79.227.39:4017   , piece_id=H2XFKXRSXEAG4WM7XDSYFLSHUOECVFXR47PBYVHW4DZLKJXWLU5A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.635153522 (301h35m55.155977267s) [noise=true , address=145.79.227.25:4619   , piece_id=3EFGZMPN4XP4F4U5OKK5VIYD44FT3WXQB73NZQL4LFYVQMTSUIVA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.639359247 (301h35m55.160182992s) [noise=true , address=145.79.227.4:4185    , piece_id=73P5QCJDK5IESMP53DQNMZHRSOHFDVYXEM6AWW7CWR7PDDKH4EXQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.642768653 (301h35m55.163592388s) [noise=true , address=145.79.227.31:4798   , piece_id=2ZEX5FUWBC2NYJJCGZ5UMPBGKKXNGXFH27CMWB7I55BWAQ4X33XQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.669518538 (301h35m55.190342283s) [noise=true , address=145.79.227.13:4804   , piece_id=GAB5UUUHLI3P6YSFVQ6HHBTPCTLQOPE6THKSHKZPSAHLI7G2QFHA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.673346913 (301h35m55.194170658s) [noise=true , address=145.79.227.26:4654   , piece_id=OYHDYSBP5LMUP6ZQONZNELXVOY53ZSJMG33ONRQLLFGYPFKR3P5Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.722674662 (301h35m55.243498407s) [noise=true , address=77.90.31.15:4701     , piece_id=XQI7GRPTXPE35WKWKEQX72RSEQ74NMF7645CXJA2E5FKWWKHEI2A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.722697015 (301h35m55.24352076s) [seg_pos={0 0}, upload=288931]: Satisfied cohort requirements after 13 pieces
May 13 08:23:44.722987831 (301h35m55.243811566s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
May 13 08:23:44.722997429 (301h35m55.243821174s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723038947 (301h35m55.243862692s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Kwa9zxc9LDp78psKPNV55aP5RYvtcQi1NjphTkXBWRugwhg7s): protocol: write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection; write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection
May 13 08:23:44.723054898 (301h35m55.243878643s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723109372 (301h35m55.243933117s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XeA4A8wbmv7wgPjRQnNbP9xgDQz4MzJH7PasaYsTrsMzTJRhC): protocol: expected piece hash; context canceled; EOF
May 13 08:23:44.723116526 (301h35m55.243940281s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723129540 (301h35m55.243953285s) [seg_pos={0 0}, upload=288931]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 13 08:23:44.723189946 (301h35m55.244013691s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 13 08:23:44.723193784 (301h35m55.244017519s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 13 08:24:49.413522848 (301h36m59.934346603s) [upload=288931]: Done uploading segments...
May 13 08:24:49.413641645 (301h36m59.93446539s) [upload=288931]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.74176211s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......]
duration:
653.25088ms
remote:
"tcp:34.4.32.59:443"
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/68a242383b75c35d
FAIL (2026-05-13 08:34:03 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 653.281509ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................]
duration:
1.892543618s
trace:
https://jaeger.dev.storj.io/trace/3821857227b01f0d
FAIL (2026-05-13 08:34:01 UTC)
orbiter warning: operation: StatBucket
uplink: bucket: metaclient: context deadline exceeded
Duration: 1.892572812s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)
antiblip:
2 out of 60 is failed (threshold=3) [..........................................X...X.............]
duration:
1.901593167s
trace:
https://jaeger.dev.storj.io/trace/6b68594ae31476df
FAIL (2026-05-13 08:33:47 UTC)
orbiter warning: 
operation: StatBucket
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
Duration: 1.901619687s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............]
duration:
5.675727942s
trace:
https://jaeger.dev.storj.io/trace/f253484fd40b3cf
FAIL (2026-05-13 08:33:42 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
52.886159919s) [upload=288931]: Starting upload
May 13 08:23:42.365411428 (301h35m52.886235173s) [upload=288931]: Done starting upload
May 13 08:23:42.365500037 (301h35m52.886323782s) [upload=288931]: Uploading segments...
May 13 08:23:42.365647579 (301h35m52.886471334s) [upload=288931]: Got next segment. Inline: false
May 13 08:23:42.365683176 (301h35m52.886506911s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 13 08:23:42.365687885 (301h35m52.88651163s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 13 08:23:42.598383896 (301h35m53.119207641s) [seg_pos={0 0}, upload=288931]: Begin upload segment...
May 13 08:23:42.598464609 (301h35m53.119288344s) [seg_pos={0 0}, upload=288931]: Done begin upload segment.
May 13 08:23:42.598489827 (301h35m53.119313562s) [noise=true , address=145.79.227.4:4185    , piece_id=73P5QCJDK5IESMP53DQNMZHRSOHFDVYXEM6AWW7CWR7PDDKH4EXQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.598584227 (301h35m53.119407972s) [noise=true , address=77.90.31.15:4701     , piece_id=XQI7GRPTXPE35WKWKEQX72RSEQ74NMF7645CXJA2E5FKWWKHEI2A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599099290 (301h35m53.119923035s) [noise=true , address=145.79.227.52:4447   , piece_id=XTJJIUVCL2WH5TJ4CZRJAUETVQIYYB55KALFX46YG2FPOMLLOPUA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599448427 (301h35m53.120272172s) [noise=true , address=145.79.227.3:4152    , piece_id=BZGI2SX3KLTQ5L74GQTXWJ3MXN5KLLFZKMPES5FRGGCLUD7XXV3A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599785570 (301h35m53.120609315s) [noise=true , address=145.79.227.7:4297    , piece_id=W2ZWVO5KC5ZJMC5LZKQ3ZBGEOBGE2Z7DN6BLRZCGPJ2LWDFEIAGA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600089149 (301h35m53.120912894s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600393219 (301h35m53.121216964s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600679686 (301h35m53.121503431s) [noise=true , address=145.79.227.6:4263    , piece_id=MPF6TBBYJ3IQVVBMS36BIV6PAITZ25AYM3TJAI2KO7XBZB2FN7WQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.601010748 (301h35m53.121834493s) [noise=true , address=145.79.227.26:4654   , piece_id=OYHDYSBP5LMUP6ZQONZNELXVOY53ZSJMG33ONRQLLFGYPFKR3P5Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.601837916 (301h35m53.122661651s) [noise=true , address=145.79.227.39:4017   , piece_id=H2XFKXRSXEAG4WM7XDSYFLSHUOECVFXR47PBYVHW4DZLKJXWLU5A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.602900945 (301h35m53.12372469s) [noise=true , address=145.79.227.45:4233   , piece_id=25YQOKEQLNYGD2DZBPWQD5HMUKBXEA2YMYVLRZWQVHCPXWKFOA3Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.603751337 (301h35m53.124575072s) [noise=true , address=145.79.227.25:4619   , piece_id=3EFGZMPN4XP4F4U5OKK5VIYD44FT3WXQB73NZQL4LFYVQMTSUIVA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.604687004 (301h35m53.125510749s) [noise=true , address=145.79.227.13:4804   , piece_id=GAB5UUUHLI3P6YSFVQ6HHBTPCTLQOPE6THKSHKZPSAHLI7G2QFHA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.605665119 (301h35m53.126488864s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.606795627 (301h35m53.127619372s) [noise=true , address=145.79.227.50:4374   , piece_id=VU7FGWLJIQ6BJWLWR4V3FZ2OOKWZ4XUXXIK6UTIOXG4JQRS5QAVQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.607929281 (301h35m53.128753026s) [noise=true , address=145.79.227.31:4798   , piece_id=2ZEX5FUWBC2NYJJCGZ5UMPBGKKXNGXFH27CMWB7I55BWAQ4X33XQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:44.318577649 (301h35m54.839401394s) [upload=288931]: Next returned nil segment
May 13 08:23:44.318598147 (301h35m54.839421892s) [upload=288931]: Waiting for error group managing segments...
May 13 08:23:44.604157373 (301h35m55.124981108s) [noise=true , address=145.79.227.50:4374   , piece_id=VU7FGWLJIQ6BJWLWR4V3FZ2OOKWZ4XUXXIK6UTIOXG4JQRS5QAVQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.616951337 (301h35m55.137775082s) [noise=true , address=145.79.227.45:4233   , piece_id=25YQOKEQLNYGD2DZBPWQD5HMUKBXEA2YMYVLRZWQVHCPXWKFOA3Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.626138077 (301h35m55.146961822s) [noise=true , address=145.79.227.7:4297    , piece_id=W2ZWVO5KC5ZJMC5LZKQ3ZBGEOBGE2Z7DN6BLRZCGPJ2LWDFEIAGA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627298322 (301h35m55.148122067s) [noise=true , address=145.79.227.3:4152    , piece_id=BZGI2SX3KLTQ5L74GQTXWJ3MXN5KLLFZKMPES5FRGGCLUD7XXV3A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627592093 (301h35m55.148415838s) [noise=true , address=145.79.227.52:4447   , piece_id=XTJJIUVCL2WH5TJ4CZRJAUETVQIYYB55KALFX46YG2FPOMLLOPUA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627893708 (301h35m55.148717453s) [noise=true , address=145.79.227.6:4263    , piece_id=MPF6TBBYJ3IQVVBMS36BIV6PAITZ25AYM3TJAI2KO7XBZB2FN7WQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.630105298 (301h35m55.150929043s) [noise=true , address=145.79.227.39:4017   , piece_id=H2XFKXRSXEAG4WM7XDSYFLSHUOECVFXR47PBYVHW4DZLKJXWLU5A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.635153522 (301h35m55.155977267s) [noise=true , address=145.79.227.25:4619   , piece_id=3EFGZMPN4XP4F4U5OKK5VIYD44FT3WXQB73NZQL4LFYVQMTSUIVA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.639359247 (301h35m55.160182992s) [noise=true , address=145.79.227.4:4185    , piece_id=73P5QCJDK5IESMP53DQNMZHRSOHFDVYXEM6AWW7CWR7PDDKH4EXQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.642768653 (301h35m55.163592388s) [noise=true , address=145.79.227.31:4798   , piece_id=2ZEX5FUWBC2NYJJCGZ5UMPBGKKXNGXFH27CMWB7I55BWAQ4X33XQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.669518538 (301h35m55.190342283s) [noise=true , address=145.79.227.13:4804   , piece_id=GAB5UUUHLI3P6YSFVQ6HHBTPCTLQOPE6THKSHKZPSAHLI7G2QFHA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.673346913 (301h35m55.194170658s) [noise=true , address=145.79.227.26:4654   , piece_id=OYHDYSBP5LMUP6ZQONZNELXVOY53ZSJMG33ONRQLLFGYPFKR3P5Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.722674662 (301h35m55.243498407s) [noise=true , address=77.90.31.15:4701     , piece_id=XQI7GRPTXPE35WKWKEQX72RSEQ74NMF7645CXJA2E5FKWWKHEI2A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.722697015 (301h35m55.24352076s) [seg_pos={0 0}, upload=288931]: Satisfied cohort requirements after 13 pieces
May 13 08:23:44.722987831 (301h35m55.243811566s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
May 13 08:23:44.722997429 (301h35m55.243821174s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723038947 (301h35m55.243862692s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Kwa9zxc9LDp78psKPNV55aP5RYvtcQi1NjphTkXBWRugwhg7s): protocol: write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection; write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection
May 13 08:23:44.723054898 (301h35m55.243878643s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723109372 (301h35m55.243933117s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XeA4A8wbmv7wgPjRQnNbP9xgDQz4MzJH7PasaYsTrsMzTJRhC): protocol: expected piece hash; context canceled; EOF
May 13 08:23:44.723116526 (301h35m55.243940281s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723129540 (301h35m55.243953285s) [seg_pos={0 0}, upload=288931]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 13 08:23:44.723189946 (301h35m55.244013691s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 13 08:23:44.723193784 (301h35m55.244017519s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 13 08:24:49.413522848 (301h36m59.934346603s) [upload=288931]: Done uploading segments...
May 13 08:24:49.413641645 (301h36m59.93446539s) [upload=288931]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.675750215s core uplink us-select-1
checkers.HTTP{"GET" "http://timesync.objectmount.io/health", ExpectedStatus: 200} failure (antiblip window 3/60): (http.timesync_objectmount_io_health)
antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............]
duration:
433.302184ms
target:
http://timesync.objectmount.io/health
trace:
https://jaeger.dev.storj.io/trace/498a8e7089f2c4a8
FAIL (2026-05-13 08:33:38 UTC)
orbiter warning: Get "http://timesync.objectmount.io/health": dial tcp 35.209.219.1:80: i/o timeout
Duration: 433.35247ms object-mount
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............]
duration:
5.611518186s
trace:
https://jaeger.dev.storj.io/trace/f253484fd40b3cf
FAIL (2026-05-13 08:33:24 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
52.886159919s) [upload=288931]: Starting upload
May 13 08:23:42.365411428 (301h35m52.886235173s) [upload=288931]: Done starting upload
May 13 08:23:42.365500037 (301h35m52.886323782s) [upload=288931]: Uploading segments...
May 13 08:23:42.365647579 (301h35m52.886471334s) [upload=288931]: Got next segment. Inline: false
May 13 08:23:42.365683176 (301h35m52.886506911s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 13 08:23:42.365687885 (301h35m52.88651163s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 13 08:23:42.598383896 (301h35m53.119207641s) [seg_pos={0 0}, upload=288931]: Begin upload segment...
May 13 08:23:42.598464609 (301h35m53.119288344s) [seg_pos={0 0}, upload=288931]: Done begin upload segment.
May 13 08:23:42.598489827 (301h35m53.119313562s) [noise=true , address=145.79.227.4:4185    , piece_id=73P5QCJDK5IESMP53DQNMZHRSOHFDVYXEM6AWW7CWR7PDDKH4EXQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.598584227 (301h35m53.119407972s) [noise=true , address=77.90.31.15:4701     , piece_id=XQI7GRPTXPE35WKWKEQX72RSEQ74NMF7645CXJA2E5FKWWKHEI2A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599099290 (301h35m53.119923035s) [noise=true , address=145.79.227.52:4447   , piece_id=XTJJIUVCL2WH5TJ4CZRJAUETVQIYYB55KALFX46YG2FPOMLLOPUA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599448427 (301h35m53.120272172s) [noise=true , address=145.79.227.3:4152    , piece_id=BZGI2SX3KLTQ5L74GQTXWJ3MXN5KLLFZKMPES5FRGGCLUD7XXV3A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599785570 (301h35m53.120609315s) [noise=true , address=145.79.227.7:4297    , piece_id=W2ZWVO5KC5ZJMC5LZKQ3ZBGEOBGE2Z7DN6BLRZCGPJ2LWDFEIAGA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600089149 (301h35m53.120912894s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600393219 (301h35m53.121216964s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600679686 (301h35m53.121503431s) [noise=true , address=145.79.227.6:4263    , piece_id=MPF6TBBYJ3IQVVBMS36BIV6PAITZ25AYM3TJAI2KO7XBZB2FN7WQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.601010748 (301h35m53.121834493s) [noise=true , address=145.79.227.26:4654   , piece_id=OYHDYSBP5LMUP6ZQONZNELXVOY53ZSJMG33ONRQLLFGYPFKR3P5Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.601837916 (301h35m53.122661651s) [noise=true , address=145.79.227.39:4017   , piece_id=H2XFKXRSXEAG4WM7XDSYFLSHUOECVFXR47PBYVHW4DZLKJXWLU5A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.602900945 (301h35m53.12372469s) [noise=true , address=145.79.227.45:4233   , piece_id=25YQOKEQLNYGD2DZBPWQD5HMUKBXEA2YMYVLRZWQVHCPXWKFOA3Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.603751337 (301h35m53.124575072s) [noise=true , address=145.79.227.25:4619   , piece_id=3EFGZMPN4XP4F4U5OKK5VIYD44FT3WXQB73NZQL4LFYVQMTSUIVA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.604687004 (301h35m53.125510749s) [noise=true , address=145.79.227.13:4804   , piece_id=GAB5UUUHLI3P6YSFVQ6HHBTPCTLQOPE6THKSHKZPSAHLI7G2QFHA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.605665119 (301h35m53.126488864s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.606795627 (301h35m53.127619372s) [noise=true , address=145.79.227.50:4374   , piece_id=VU7FGWLJIQ6BJWLWR4V3FZ2OOKWZ4XUXXIK6UTIOXG4JQRS5QAVQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.607929281 (301h35m53.128753026s) [noise=true , address=145.79.227.31:4798   , piece_id=2ZEX5FUWBC2NYJJCGZ5UMPBGKKXNGXFH27CMWB7I55BWAQ4X33XQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:44.318577649 (301h35m54.839401394s) [upload=288931]: Next returned nil segment
May 13 08:23:44.318598147 (301h35m54.839421892s) [upload=288931]: Waiting for error group managing segments...
May 13 08:23:44.604157373 (301h35m55.124981108s) [noise=true , address=145.79.227.50:4374   , piece_id=VU7FGWLJIQ6BJWLWR4V3FZ2OOKWZ4XUXXIK6UTIOXG4JQRS5QAVQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.616951337 (301h35m55.137775082s) [noise=true , address=145.79.227.45:4233   , piece_id=25YQOKEQLNYGD2DZBPWQD5HMUKBXEA2YMYVLRZWQVHCPXWKFOA3Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.626138077 (301h35m55.146961822s) [noise=true , address=145.79.227.7:4297    , piece_id=W2ZWVO5KC5ZJMC5LZKQ3ZBGEOBGE2Z7DN6BLRZCGPJ2LWDFEIAGA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627298322 (301h35m55.148122067s) [noise=true , address=145.79.227.3:4152    , piece_id=BZGI2SX3KLTQ5L74GQTXWJ3MXN5KLLFZKMPES5FRGGCLUD7XXV3A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627592093 (301h35m55.148415838s) [noise=true , address=145.79.227.52:4447   , piece_id=XTJJIUVCL2WH5TJ4CZRJAUETVQIYYB55KALFX46YG2FPOMLLOPUA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627893708 (301h35m55.148717453s) [noise=true , address=145.79.227.6:4263    , piece_id=MPF6TBBYJ3IQVVBMS36BIV6PAITZ25AYM3TJAI2KO7XBZB2FN7WQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.630105298 (301h35m55.150929043s) [noise=true , address=145.79.227.39:4017   , piece_id=H2XFKXRSXEAG4WM7XDSYFLSHUOECVFXR47PBYVHW4DZLKJXWLU5A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.635153522 (301h35m55.155977267s) [noise=true , address=145.79.227.25:4619   , piece_id=3EFGZMPN4XP4F4U5OKK5VIYD44FT3WXQB73NZQL4LFYVQMTSUIVA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.639359247 (301h35m55.160182992s) [noise=true , address=145.79.227.4:4185    , piece_id=73P5QCJDK5IESMP53DQNMZHRSOHFDVYXEM6AWW7CWR7PDDKH4EXQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.642768653 (301h35m55.163592388s) [noise=true , address=145.79.227.31:4798   , piece_id=2ZEX5FUWBC2NYJJCGZ5UMPBGKKXNGXFH27CMWB7I55BWAQ4X33XQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.669518538 (301h35m55.190342283s) [noise=true , address=145.79.227.13:4804   , piece_id=GAB5UUUHLI3P6YSFVQ6HHBTPCTLQOPE6THKSHKZPSAHLI7G2QFHA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.673346913 (301h35m55.194170658s) [noise=true , address=145.79.227.26:4654   , piece_id=OYHDYSBP5LMUP6ZQONZNELXVOY53ZSJMG33ONRQLLFGYPFKR3P5Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.722674662 (301h35m55.243498407s) [noise=true , address=77.90.31.15:4701     , piece_id=XQI7GRPTXPE35WKWKEQX72RSEQ74NMF7645CXJA2E5FKWWKHEI2A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.722697015 (301h35m55.24352076s) [seg_pos={0 0}, upload=288931]: Satisfied cohort requirements after 13 pieces
May 13 08:23:44.722987831 (301h35m55.243811566s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
May 13 08:23:44.722997429 (301h35m55.243821174s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723038947 (301h35m55.243862692s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Kwa9zxc9LDp78psKPNV55aP5RYvtcQi1NjphTkXBWRugwhg7s): protocol: write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection; write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection
May 13 08:23:44.723054898 (301h35m55.243878643s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723109372 (301h35m55.243933117s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XeA4A8wbmv7wgPjRQnNbP9xgDQz4MzJH7PasaYsTrsMzTJRhC): protocol: expected piece hash; context canceled; EOF
May 13 08:23:44.723116526 (301h35m55.243940281s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723129540 (301h35m55.243953285s) [seg_pos={0 0}, upload=288931]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 13 08:23:44.723189946 (301h35m55.244013691s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 13 08:23:44.723193784 (301h35m55.244017519s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 13 08:24:49.413522848 (301h36m59.934346603s) [upload=288931]: Done uploading segments...
May 13 08:24:49.413641645 (301h36m59.93446539s) [upload=288931]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.611561468s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......]
duration:
652.591542ms
remote:
"tcp:34.4.32.59:443"
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/68a242383b75c35d
FAIL (2026-05-13 08:32:58 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 652.60622ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)
antiblip:
2 out of 60 is failed (threshold=3) [...........................................X...X............]
duration:
1.886766023s
trace:
https://jaeger.dev.storj.io/trace/6b68594ae31476df
FAIL (2026-05-13 08:32:49 UTC)
orbiter warning: 
operation: StatBucket
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
Duration: 1.88678526s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................]
duration:
1.889052126s
trace:
https://jaeger.dev.storj.io/trace/3821857227b01f0d
FAIL (2026-05-13 08:32:47 UTC)
orbiter warning: operation: StatBucket
uplink: bucket: metaclient: context deadline exceeded
Duration: 1.889176604s core uplink us1
checkers.HTTP{"GET" "http://timesync.objectmount.io/health", ExpectedStatus: 200} failure (antiblip window 3/60): (http.timesync_objectmount_io_health)
antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............]
duration:
424.083133ms
target:
http://timesync.objectmount.io/health
trace:
https://jaeger.dev.storj.io/trace/498a8e7089f2c4a8
FAIL (2026-05-13 08:32:30 UTC)
orbiter warning: Get "http://timesync.objectmount.io/health": dial tcp 35.209.219.1:80: i/o timeout
Duration: 424.11269ms object-mount
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)
antiblip:
2 out of 60 is failed (threshold=3) [............................................X...X...........]
duration:
1.903972876s
trace:
https://jaeger.dev.storj.io/trace/6b68594ae31476df
FAIL (2026-05-13 08:32:16 UTC)
orbiter warning: 
operation: StatBucket
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
Duration: 1.903989318s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........]
duration:
5.551853869s
trace:
https://jaeger.dev.storj.io/trace/f253484fd40b3cf
FAIL (2026-05-13 08:32:12 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
52.886159919s) [upload=288931]: Starting upload
May 13 08:23:42.365411428 (301h35m52.886235173s) [upload=288931]: Done starting upload
May 13 08:23:42.365500037 (301h35m52.886323782s) [upload=288931]: Uploading segments...
May 13 08:23:42.365647579 (301h35m52.886471334s) [upload=288931]: Got next segment. Inline: false
May 13 08:23:42.365683176 (301h35m52.886506911s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 13 08:23:42.365687885 (301h35m52.88651163s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 13 08:23:42.598383896 (301h35m53.119207641s) [seg_pos={0 0}, upload=288931]: Begin upload segment...
May 13 08:23:42.598464609 (301h35m53.119288344s) [seg_pos={0 0}, upload=288931]: Done begin upload segment.
May 13 08:23:42.598489827 (301h35m53.119313562s) [noise=true , address=145.79.227.4:4185    , piece_id=73P5QCJDK5IESMP53DQNMZHRSOHFDVYXEM6AWW7CWR7PDDKH4EXQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.598584227 (301h35m53.119407972s) [noise=true , address=77.90.31.15:4701     , piece_id=XQI7GRPTXPE35WKWKEQX72RSEQ74NMF7645CXJA2E5FKWWKHEI2A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599099290 (301h35m53.119923035s) [noise=true , address=145.79.227.52:4447   , piece_id=XTJJIUVCL2WH5TJ4CZRJAUETVQIYYB55KALFX46YG2FPOMLLOPUA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599448427 (301h35m53.120272172s) [noise=true , address=145.79.227.3:4152    , piece_id=BZGI2SX3KLTQ5L74GQTXWJ3MXN5KLLFZKMPES5FRGGCLUD7XXV3A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599785570 (301h35m53.120609315s) [noise=true , address=145.79.227.7:4297    , piece_id=W2ZWVO5KC5ZJMC5LZKQ3ZBGEOBGE2Z7DN6BLRZCGPJ2LWDFEIAGA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600089149 (301h35m53.120912894s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600393219 (301h35m53.121216964s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600679686 (301h35m53.121503431s) [noise=true , address=145.79.227.6:4263    , piece_id=MPF6TBBYJ3IQVVBMS36BIV6PAITZ25AYM3TJAI2KO7XBZB2FN7WQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.601010748 (301h35m53.121834493s) [noise=true , address=145.79.227.26:4654   , piece_id=OYHDYSBP5LMUP6ZQONZNELXVOY53ZSJMG33ONRQLLFGYPFKR3P5Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.601837916 (301h35m53.122661651s) [noise=true , address=145.79.227.39:4017   , piece_id=H2XFKXRSXEAG4WM7XDSYFLSHUOECVFXR47PBYVHW4DZLKJXWLU5A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.602900945 (301h35m53.12372469s) [noise=true , address=145.79.227.45:4233   , piece_id=25YQOKEQLNYGD2DZBPWQD5HMUKBXEA2YMYVLRZWQVHCPXWKFOA3Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.603751337 (301h35m53.124575072s) [noise=true , address=145.79.227.25:4619   , piece_id=3EFGZMPN4XP4F4U5OKK5VIYD44FT3WXQB73NZQL4LFYVQMTSUIVA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.604687004 (301h35m53.125510749s) [noise=true , address=145.79.227.13:4804   , piece_id=GAB5UUUHLI3P6YSFVQ6HHBTPCTLQOPE6THKSHKZPSAHLI7G2QFHA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.605665119 (301h35m53.126488864s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.606795627 (301h35m53.127619372s) [noise=true , address=145.79.227.50:4374   , piece_id=VU7FGWLJIQ6BJWLWR4V3FZ2OOKWZ4XUXXIK6UTIOXG4JQRS5QAVQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.607929281 (301h35m53.128753026s) [noise=true , address=145.79.227.31:4798   , piece_id=2ZEX5FUWBC2NYJJCGZ5UMPBGKKXNGXFH27CMWB7I55BWAQ4X33XQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:44.318577649 (301h35m54.839401394s) [upload=288931]: Next returned nil segment
May 13 08:23:44.318598147 (301h35m54.839421892s) [upload=288931]: Waiting for error group managing segments...
May 13 08:23:44.604157373 (301h35m55.124981108s) [noise=true , address=145.79.227.50:4374   , piece_id=VU7FGWLJIQ6BJWLWR4V3FZ2OOKWZ4XUXXIK6UTIOXG4JQRS5QAVQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.616951337 (301h35m55.137775082s) [noise=true , address=145.79.227.45:4233   , piece_id=25YQOKEQLNYGD2DZBPWQD5HMUKBXEA2YMYVLRZWQVHCPXWKFOA3Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.626138077 (301h35m55.146961822s) [noise=true , address=145.79.227.7:4297    , piece_id=W2ZWVO5KC5ZJMC5LZKQ3ZBGEOBGE2Z7DN6BLRZCGPJ2LWDFEIAGA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627298322 (301h35m55.148122067s) [noise=true , address=145.79.227.3:4152    , piece_id=BZGI2SX3KLTQ5L74GQTXWJ3MXN5KLLFZKMPES5FRGGCLUD7XXV3A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627592093 (301h35m55.148415838s) [noise=true , address=145.79.227.52:4447   , piece_id=XTJJIUVCL2WH5TJ4CZRJAUETVQIYYB55KALFX46YG2FPOMLLOPUA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627893708 (301h35m55.148717453s) [noise=true , address=145.79.227.6:4263    , piece_id=MPF6TBBYJ3IQVVBMS36BIV6PAITZ25AYM3TJAI2KO7XBZB2FN7WQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.630105298 (301h35m55.150929043s) [noise=true , address=145.79.227.39:4017   , piece_id=H2XFKXRSXEAG4WM7XDSYFLSHUOECVFXR47PBYVHW4DZLKJXWLU5A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.635153522 (301h35m55.155977267s) [noise=true , address=145.79.227.25:4619   , piece_id=3EFGZMPN4XP4F4U5OKK5VIYD44FT3WXQB73NZQL4LFYVQMTSUIVA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.639359247 (301h35m55.160182992s) [noise=true , address=145.79.227.4:4185    , piece_id=73P5QCJDK5IESMP53DQNMZHRSOHFDVYXEM6AWW7CWR7PDDKH4EXQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.642768653 (301h35m55.163592388s) [noise=true , address=145.79.227.31:4798   , piece_id=2ZEX5FUWBC2NYJJCGZ5UMPBGKKXNGXFH27CMWB7I55BWAQ4X33XQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.669518538 (301h35m55.190342283s) [noise=true , address=145.79.227.13:4804   , piece_id=GAB5UUUHLI3P6YSFVQ6HHBTPCTLQOPE6THKSHKZPSAHLI7G2QFHA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.673346913 (301h35m55.194170658s) [noise=true , address=145.79.227.26:4654   , piece_id=OYHDYSBP5LMUP6ZQONZNELXVOY53ZSJMG33ONRQLLFGYPFKR3P5Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.722674662 (301h35m55.243498407s) [noise=true , address=77.90.31.15:4701     , piece_id=XQI7GRPTXPE35WKWKEQX72RSEQ74NMF7645CXJA2E5FKWWKHEI2A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.722697015 (301h35m55.24352076s) [seg_pos={0 0}, upload=288931]: Satisfied cohort requirements after 13 pieces
May 13 08:23:44.722987831 (301h35m55.243811566s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
May 13 08:23:44.722997429 (301h35m55.243821174s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723038947 (301h35m55.243862692s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Kwa9zxc9LDp78psKPNV55aP5RYvtcQi1NjphTkXBWRugwhg7s): protocol: write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection; write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection
May 13 08:23:44.723054898 (301h35m55.243878643s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723109372 (301h35m55.243933117s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XeA4A8wbmv7wgPjRQnNbP9xgDQz4MzJH7PasaYsTrsMzTJRhC): protocol: expected piece hash; context canceled; EOF
May 13 08:23:44.723116526 (301h35m55.243940281s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723129540 (301h35m55.243953285s) [seg_pos={0 0}, upload=288931]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 13 08:23:44.723189946 (301h35m55.244013691s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 13 08:23:44.723193784 (301h35m55.244017519s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 13 08:24:49.413522848 (301h36m59.934346603s) [upload=288931]: Done uploading segments...
May 13 08:24:49.413641645 (301h36m59.93446539s) [upload=288931]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.551942267s core uplink us-select-1
checkers.HTTP{"GET" "http://timesync.objectmount.io/health", ExpectedStatus: 200} failure (antiblip window 3/60): (http.timesync_objectmount_io_health)
antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............]
duration:
423.973624ms
target:
http://timesync.objectmount.io/health
trace:
https://jaeger.dev.storj.io/trace/498a8e7089f2c4a8
FAIL (2026-05-13 08:32:06 UTC)
orbiter warning: Get "http://timesync.objectmount.io/health": dial tcp 35.209.219.1:80: i/o timeout
Duration: 423.99735ms object-mount
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................]
duration:
1.906262857s
trace:
https://jaeger.dev.storj.io/trace/3821857227b01f0d
FAIL (2026-05-13 08:31:59 UTC)
orbiter warning: operation: StatBucket
uplink: bucket: metaclient: context deadline exceeded
Duration: 1.906291302s core uplink us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....]
duration:
658.875974ms
remote:
"tcp:34.4.32.59:443"
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/68a242383b75c35d
FAIL (2026-05-13 08:31:58 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 658.900871ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........]
duration:
7.424061593s
trace:
https://jaeger.dev.storj.io/trace/f253484fd40b3cf
FAIL (2026-05-13 08:31:43 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
52.886159919s) [upload=288931]: Starting upload
May 13 08:23:42.365411428 (301h35m52.886235173s) [upload=288931]: Done starting upload
May 13 08:23:42.365500037 (301h35m52.886323782s) [upload=288931]: Uploading segments...
May 13 08:23:42.365647579 (301h35m52.886471334s) [upload=288931]: Got next segment. Inline: false
May 13 08:23:42.365683176 (301h35m52.886506911s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 13 08:23:42.365687885 (301h35m52.88651163s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 13 08:23:42.598383896 (301h35m53.119207641s) [seg_pos={0 0}, upload=288931]: Begin upload segment...
May 13 08:23:42.598464609 (301h35m53.119288344s) [seg_pos={0 0}, upload=288931]: Done begin upload segment.
May 13 08:23:42.598489827 (301h35m53.119313562s) [noise=true , address=145.79.227.4:4185    , piece_id=73P5QCJDK5IESMP53DQNMZHRSOHFDVYXEM6AWW7CWR7PDDKH4EXQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.598584227 (301h35m53.119407972s) [noise=true , address=77.90.31.15:4701     , piece_id=XQI7GRPTXPE35WKWKEQX72RSEQ74NMF7645CXJA2E5FKWWKHEI2A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599099290 (301h35m53.119923035s) [noise=true , address=145.79.227.52:4447   , piece_id=XTJJIUVCL2WH5TJ4CZRJAUETVQIYYB55KALFX46YG2FPOMLLOPUA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599448427 (301h35m53.120272172s) [noise=true , address=145.79.227.3:4152    , piece_id=BZGI2SX3KLTQ5L74GQTXWJ3MXN5KLLFZKMPES5FRGGCLUD7XXV3A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.599785570 (301h35m53.120609315s) [noise=true , address=145.79.227.7:4297    , piece_id=W2ZWVO5KC5ZJMC5LZKQ3ZBGEOBGE2Z7DN6BLRZCGPJ2LWDFEIAGA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600089149 (301h35m53.120912894s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600393219 (301h35m53.121216964s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.600679686 (301h35m53.121503431s) [noise=true , address=145.79.227.6:4263    , piece_id=MPF6TBBYJ3IQVVBMS36BIV6PAITZ25AYM3TJAI2KO7XBZB2FN7WQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.601010748 (301h35m53.121834493s) [noise=true , address=145.79.227.26:4654   , piece_id=OYHDYSBP5LMUP6ZQONZNELXVOY53ZSJMG33ONRQLLFGYPFKR3P5Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.601837916 (301h35m53.122661651s) [noise=true , address=145.79.227.39:4017   , piece_id=H2XFKXRSXEAG4WM7XDSYFLSHUOECVFXR47PBYVHW4DZLKJXWLU5A, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.602900945 (301h35m53.12372469s) [noise=true , address=145.79.227.45:4233   , piece_id=25YQOKEQLNYGD2DZBPWQD5HMUKBXEA2YMYVLRZWQVHCPXWKFOA3Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.603751337 (301h35m53.124575072s) [noise=true , address=145.79.227.25:4619   , piece_id=3EFGZMPN4XP4F4U5OKK5VIYD44FT3WXQB73NZQL4LFYVQMTSUIVA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.604687004 (301h35m53.125510749s) [noise=true , address=145.79.227.13:4804   , piece_id=GAB5UUUHLI3P6YSFVQ6HHBTPCTLQOPE6THKSHKZPSAHLI7G2QFHA, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.605665119 (301h35m53.126488864s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.606795627 (301h35m53.127619372s) [noise=true , address=145.79.227.50:4374   , piece_id=VU7FGWLJIQ6BJWLWR4V3FZ2OOKWZ4XUXXIK6UTIOXG4JQRS5QAVQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:42.607929281 (301h35m53.128753026s) [noise=true , address=145.79.227.31:4798   , piece_id=2ZEX5FUWBC2NYJJCGZ5UMPBGKKXNGXFH27CMWB7I55BWAQ4X33XQ, seg_pos={0 0}, upload=288931]: Uploading piece...
May 13 08:23:44.318577649 (301h35m54.839401394s) [upload=288931]: Next returned nil segment
May 13 08:23:44.318598147 (301h35m54.839421892s) [upload=288931]: Waiting for error group managing segments...
May 13 08:23:44.604157373 (301h35m55.124981108s) [noise=true , address=145.79.227.50:4374   , piece_id=VU7FGWLJIQ6BJWLWR4V3FZ2OOKWZ4XUXXIK6UTIOXG4JQRS5QAVQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.616951337 (301h35m55.137775082s) [noise=true , address=145.79.227.45:4233   , piece_id=25YQOKEQLNYGD2DZBPWQD5HMUKBXEA2YMYVLRZWQVHCPXWKFOA3Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.626138077 (301h35m55.146961822s) [noise=true , address=145.79.227.7:4297    , piece_id=W2ZWVO5KC5ZJMC5LZKQ3ZBGEOBGE2Z7DN6BLRZCGPJ2LWDFEIAGA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627298322 (301h35m55.148122067s) [noise=true , address=145.79.227.3:4152    , piece_id=BZGI2SX3KLTQ5L74GQTXWJ3MXN5KLLFZKMPES5FRGGCLUD7XXV3A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627592093 (301h35m55.148415838s) [noise=true , address=145.79.227.52:4447   , piece_id=XTJJIUVCL2WH5TJ4CZRJAUETVQIYYB55KALFX46YG2FPOMLLOPUA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.627893708 (301h35m55.148717453s) [noise=true , address=145.79.227.6:4263    , piece_id=MPF6TBBYJ3IQVVBMS36BIV6PAITZ25AYM3TJAI2KO7XBZB2FN7WQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.630105298 (301h35m55.150929043s) [noise=true , address=145.79.227.39:4017   , piece_id=H2XFKXRSXEAG4WM7XDSYFLSHUOECVFXR47PBYVHW4DZLKJXWLU5A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.635153522 (301h35m55.155977267s) [noise=true , address=145.79.227.25:4619   , piece_id=3EFGZMPN4XP4F4U5OKK5VIYD44FT3WXQB73NZQL4LFYVQMTSUIVA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.639359247 (301h35m55.160182992s) [noise=true , address=145.79.227.4:4185    , piece_id=73P5QCJDK5IESMP53DQNMZHRSOHFDVYXEM6AWW7CWR7PDDKH4EXQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.642768653 (301h35m55.163592388s) [noise=true , address=145.79.227.31:4798   , piece_id=2ZEX5FUWBC2NYJJCGZ5UMPBGKKXNGXFH27CMWB7I55BWAQ4X33XQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.669518538 (301h35m55.190342283s) [noise=true , address=145.79.227.13:4804   , piece_id=GAB5UUUHLI3P6YSFVQ6HHBTPCTLQOPE6THKSHKZPSAHLI7G2QFHA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.673346913 (301h35m55.194170658s) [noise=true , address=145.79.227.26:4654   , piece_id=OYHDYSBP5LMUP6ZQONZNELXVOY53ZSJMG33ONRQLLFGYPFKR3P5Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.722674662 (301h35m55.243498407s) [noise=true , address=77.90.31.15:4701     , piece_id=XQI7GRPTXPE35WKWKEQX72RSEQ74NMF7645CXJA2E5FKWWKHEI2A, seg_pos={0 0}, upload=288931]: Done uploading piece. err: <nil>
May 13 08:23:44.722697015 (301h35m55.24352076s) [seg_pos={0 0}, upload=288931]: Satisfied cohort requirements after 13 pieces
May 13 08:23:44.722987831 (301h35m55.243811566s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF
May 13 08:23:44.722997429 (301h35m55.243821174s) [noise=true , address=77.90.31.13:4626     , piece_id=YS5MLLTJQBYQ6VCEKMYUNBVEIKVQAC7L22PRY2BRMLPA25IR3D7Q, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723038947 (301h35m55.243862692s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Kwa9zxc9LDp78psKPNV55aP5RYvtcQi1NjphTkXBWRugwhg7s): protocol: write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection; write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37572->45.91.116.4:4972: use of closed network connection
May 13 08:23:44.723054898 (301h35m55.243878643s) [noise=true , address=45.91.116.4:4972     , piece_id=GHKQU3ZRVRBRIQ3SYS42DGTC2K3VQ3UHAGLBKST7QZYAH5LSEXLA, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723109372 (301h35m55.243933117s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XeA4A8wbmv7wgPjRQnNbP9xgDQz4MzJH7PasaYsTrsMzTJRhC): protocol: expected piece hash; context canceled; EOF
May 13 08:23:44.723116526 (301h35m55.243940281s) [noise=true , address=145.79.227.49:4340   , piece_id=VH6N2FJ76D6UQS3DBHRDBXVMMMHVL73MS5MMLF4X6ITFGIRZYMZQ, seg_pos={0 0}, upload=288931]: Long tail cancellation due to optimal threshold
May 13 08:23:44.723129540 (301h35m55.243953285s) [seg_pos={0 0}, upload=288931]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 13 08:23:44.723189946 (301h35m55.244013691s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 13 08:23:44.723193784 (301h35m55.244017519s) [upload=288931]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 13 08:24:49.413522848 (301h36m59.934346603s) [upload=288931]: Done uploading segments...
May 13 08:24:49.413641645 (301h36m59.93446539s) [upload=288931]: Upload finished. err: metaclient: context deadline exceeded
Duration: 7.424087582s core uplink us-select-1
checkers.HTTP{"GET" "http://timesync.objectmount.io/health", ExpectedStatus: 200} failure (antiblip window 3/60): (http.timesync_objectmount_io_health)
antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........]
duration:
424.033477ms
target:
http://timesync.objectmount.io/health
trace:
https://jaeger.dev.storj.io/trace/498a8e7089f2c4a8
FAIL (2026-05-13 08:31:28 UTC)
orbiter warning: Get "http://timesync.objectmount.io/health": dial tcp 35.209.219.1:80: i/o timeout
Duration: 424.062343ms object-mount
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)
antiblip:
2 out of 60 is failed (threshold=3) [.............................................X...X..........]
duration:
1.912922247s
trace:
https://jaeger.dev.storj.io/trace/6b68594ae31476df
FAIL (2026-05-13 08:31:25 UTC)
orbiter warning: 
operation: StatBucket
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
Duration: 1.912949378s core uplink us-select-1