Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
661.79303ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7e710747942acdb2
FAIL (2026-05-25 18:07:22 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 661.809553ms
core
satellite
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
11 out of 60 is failed (threshold=3) [....X................X....X.......X......X..XX.XX.X....X....] duration:
6.604653904s trace:
https://jaeger.dev.storj.io/trace/fac1154c0207b7
FAIL (2026-05-25 18:07:05 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.604666829s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
11 out of 60 is failed (threshold=3) [.....X................X....X.......X......X..XX.XX.X....X...] duration:
55.345941641s trace:
https://jaeger.dev.storj.io/trace/fac1154c0207b7
FAIL (2026-05-25 18:06:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 55.345971698s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [....X..........................................X............] duration:
3.721679396s trace:
https://jaeger.dev.storj.io/trace/14eb8131f680ed75
FAIL (2026-05-25 18:06:57 UTC)
orbiter warning: context deadline exceeded
Duration: 3.721703001s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
663.160583ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7e710747942acdb2
FAIL (2026-05-25 18:06:12 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 663.181252ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
4.054969351s trace:
https://jaeger.dev.storj.io/trace/a5682f02df33c7c
FAIL (2026-05-25 18:06:01 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ZCYA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543483940 (15h48m30.326824029s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543527172 (15h48m30.326867271s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543870378 (15h48m30.327210477s) [noise=true , address=212.102.52.216:4408 , piece_id=EPE6MXFOZTO5PT3SLA3U6HI6UJWMNBPWAPI7LLGPB35CRKB3AZOA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.544411289 (15h48m30.327751388s) [noise=true , address=212.102.52.214:4364 , piece_id=MU7IJZCB752M4M7LFCEJZCDEJJY6S5M63KH4YBSIIQN5LPHW6YQA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.544595600 (15h48m30.327935699s) [noise=true , address=212.102.52.215:4376 , piece_id=25KT756B5NJCE44RWE22WJ7O5VXP7VJR6YO3M3LTRFX3CIYPVUQA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.545402781 (15h48m30.32874288s) [noise=true , address=212.102.52.215:4382 , piece_id=JCRZDFY2O6CLCTURWS7KRMWQSDNOURAA3GLPKB4P2QO3UUNB26SA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.545463316 (15h48m30.328803415s) [noise=true , address=212.102.52.214:4360 , piece_id=73PXBGVXO5KGNJDVS75DBD56COZQYSP26RBS5GND2BLG6JQLMXZA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546346461 (15h48m30.32968656s) [noise=true , address=212.102.52.216:4402 , piece_id=B5JT2RXBOTI4IUOCRZKPKA6LY6C3IGQGHWWWOXJ5FCYC64EOBIEQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546403570 (15h48m30.329743669s) [noise=true , address=212.102.52.217:4424 , piece_id=FLGYPCAZR5EF3FDQ5MT7CHBA5HDLM4CQOS6HCPVML2XKRLFGOGGQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546993276 (15h48m30.330333375s) [noise=true , address=212.102.52.215:4386 , piece_id=SJIDT4PIAIMA7QFS64FI6FMNRRO6Q5S4PVQGUKPJQBPJ7N7ZFL2A, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.548021087 (15h48m30.331361186s) [noise=true , address=212.102.52.216:4396 , piece_id=RYO7RQ26ZTRMDK22XCED2JM2K7WPEI57X25HMT5LQ73RIE7WTTYA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.548283338 (15h48m30.331623437s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.549042276 (15h48m30.332382365s) [noise=true , address=212.102.52.213:4336 , piece_id=MGI6NLZCQPDNW5I42M4NL5FWKWNOU7RFW4TZJG2LTRZATP2C742Q, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.549395759 (15h48m30.332735858s) [noise=true , address=212.102.52.217:4435 , piece_id=RNBPLJS4P5H4B24FX7XIFKRUUAWTH3PFEA3JOYAS4LCNWUUKGSBQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.550129380 (15h48m30.333469479s) [noise=true , address=212.102.52.217:4426 , piece_id=EXMYVMU7ZI4ZVELF2EMJNOZGDZAB4QZLJZ3M5DBNKIP433NFCUTA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:45.613527985 (15h48m31.396868084s) [upload=15135]: Next returned nil segment
May 25 17:52:45.613540610 (15h48m31.396880709s) [upload=15135]: Waiting for error group managing segments...
May 25 17:52:45.894196156 (15h48m31.677536255s) [noise=true , address=212.102.52.216:4394 , piece_id=4P3VUGG3TJGW6YENC6WC4J2EHG6TQNDMPXUHROT5BHRU3246ZCYA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.902564564 (15h48m31.685904663s) [noise=true , address=212.102.52.213:4336 , piece_id=MGI6NLZCQPDNW5I42M4NL5FWKWNOU7RFW4TZJG2LTRZATP2C742Q, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.902884974 (15h48m31.686225073s) [noise=true , address=212.102.52.216:4396 , piece_id=RYO7RQ26ZTRMDK22XCED2JM2K7WPEI57X25HMT5LQ73RIE7WTTYA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.903550163 (15h48m31.686890263s) [noise=true , address=212.102.52.216:4402 , piece_id=B5JT2RXBOTI4IUOCRZKPKA6LY6C3IGQGHWWWOXJ5FCYC64EOBIEQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.913910772 (15h48m31.697250881s) [noise=true , address=212.102.52.213:4333 , piece_id=TJKMUCX6JFTHDWN4WFBSLPQXI2PQFVZHNTXMOG25UYZOCTIE6LMQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.917060331 (15h48m31.70040043s) [noise=true , address=212.102.52.214:4360 , piece_id=73PXBGVXO5KGNJDVS75DBD56COZQYSP26RBS5GND2BLG6JQLMXZA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.918744335 (15h48m31.702084434s) [noise=true , address=212.102.52.215:4386 , piece_id=SJIDT4PIAIMA7QFS64FI6FMNRRO6Q5S4PVQGUKPJQBPJ7N7ZFL2A, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.924271771 (15h48m31.70761187s) [noise=true , address=212.102.52.216:4408 , piece_id=EPE6MXFOZTO5PT3SLA3U6HI6UJWMNBPWAPI7LLGPB35CRKB3AZOA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.925897152 (15h48m31.709237251s) [noise=true , address=212.102.52.215:4382 , piece_id=JCRZDFY2O6CLCTURWS7KRMWQSDNOURAA3GLPKB4P2QO3UUNB26SA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.926060985 (15h48m31.709401084s) [noise=true , address=212.102.52.217:4435 , piece_id=RNBPLJS4P5H4B24FX7XIFKRUUAWTH3PFEA3JOYAS4LCNWUUKGSBQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.927768653 (15h48m31.711108752s) [noise=true , address=212.102.52.215:4376 , piece_id=25KT756B5NJCE44RWE22WJ7O5VXP7VJR6YO3M3LTRFX3CIYPVUQA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.929649962 (15h48m31.712990061s) [noise=true , address=212.102.52.214:4364 , piece_id=MU7IJZCB752M4M7LFCEJZCDEJJY6S5M63KH4YBSIIQN5LPHW6YQA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.957068207 (15h48m31.740408307s) [noise=true , address=212.102.52.215:4380 , piece_id=2S2FDBIAGAP7AMF3CL54BECVIGDEXDDG5OHT23FJKWPSVOUPN6RA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.957729358 (15h48m31.741069457s) [noise=true , address=212.102.52.213:4324 , piece_id=YONLVC2ARQPU3ZJOSJSG2WX6SOA65DJYA5GMIEQ5ZJ5DUPR6UEFQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959196257 (15h48m31.742536356s) [noise=true , address=212.102.52.217:4426 , piece_id=EXMYVMU7ZI4ZVELF2EMJNOZGDZAB4QZLJZ3M5DBNKIP433NFCUTA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959726810 (15h48m31.743066909s) [noise=true , address=212.102.52.217:4424 , piece_id=FLGYPCAZR5EF3FDQ5MT7CHBA5HDLM4CQOS6HCPVML2XKRLFGOGGQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959735406 (15h48m31.743075505s) [seg_pos={0 0}, upload=15135]: Satisfied cohort requirements after 16 pieces
May 25 17:52:45.959899980 (15h48m31.743240079s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MbNn3rECMrMhp1uJJRHy8v3DZkKaBEcZGwPMGumvPUvvKu5gM): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959906273 (15h48m31.743246372s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.959965084 (15h48m31.743305183s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eyfVMwFVb4Q5nbc54PSLebuyWfm2rCgCfGrNc6AovSWqe6Upn): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959969824 (15h48m31.743309923s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.959978169 (15h48m31.743318269s) [noise=true , address=212.102.52.214:4356 , piece_id=UAHRGRD6YLJZOMHHBNHHONX5UZBS5W5MJETFZJZMBBP3LPJ3CP5A, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1263MPtQZy9iB1pxhBYqZ9tgqitVmjm2X7gyehcdSRja9kWfPKd): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959986315 (15h48m31.743326414s) [noise=true , address=212.102.52.214:4356 , piece_id=UAHRGRD6YLJZOMHHBNHHONX5UZBS5W5MJETFZJZMBBP3LPJ3CP5A, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.960111764 (15h48m31.743451863s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Tyfw7Bp6ma6jTKyBJaSfr5u8w2CwExJVAwkeabjMnsHWWsDdf): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.960116693 (15h48m31.743456792s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.960130370 (15h48m31.743470469s) [seg_pos={0 0}, upload=15135]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 17:52:45.960200974 (15h48m31.743541073s) [upload=15135]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 17:52:45.960205563 (15h48m31.743545662s) [upload=15135]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 17:52:48.787245030 (15h48m34.570585129s) [upload=15135]: Done uploading segments...
May 25 17:52:48.787262443 (15h48m34.570602542s) [upload=15135]: Upload finished. err: <nil>
Duration: 4.054991472s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.....X..........................................X...........] duration:
4.032590099s trace:
https://jaeger.dev.storj.io/trace/14eb8131f680ed75
FAIL (2026-05-25 18:05:54 UTC)
orbiter warning: context deadline exceeded
Duration: 4.032629804s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
660.52277ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7e710747942acdb2
FAIL (2026-05-25 18:05:50 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 660.538871ms
core
satellite
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
11 out of 60 is failed (threshold=3) [......X................X....X.......X......X..XX.XX.X....X..] duration:
6.800416067s trace:
https://jaeger.dev.storj.io/trace/fac1154c0207b7
FAIL (2026-05-25 18:05:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.800429031s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
12 out of 60 is failed (threshold=3) [X......X................X....X.......X......X..XX.XX.X....X.] duration:
59.230749209s trace:
https://jaeger.dev.storj.io/trace/fac1154c0207b7
FAIL (2026-05-25 18:05:12 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 59.230788725s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [......X..........................................X..........] duration:
3.764358529s trace:
https://jaeger.dev.storj.io/trace/14eb8131f680ed75
FAIL (2026-05-25 18:05:11 UTC)
orbiter warning: context deadline exceeded
Duration: 3.764387035s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
5.056547893s trace:
https://jaeger.dev.storj.io/trace/a5682f02df33c7c
FAIL (2026-05-25 18:05:09 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ZCYA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543483940 (15h48m30.326824029s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543527172 (15h48m30.326867271s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543870378 (15h48m30.327210477s) [noise=true , address=212.102.52.216:4408 , piece_id=EPE6MXFOZTO5PT3SLA3U6HI6UJWMNBPWAPI7LLGPB35CRKB3AZOA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.544411289 (15h48m30.327751388s) [noise=true , address=212.102.52.214:4364 , piece_id=MU7IJZCB752M4M7LFCEJZCDEJJY6S5M63KH4YBSIIQN5LPHW6YQA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.544595600 (15h48m30.327935699s) [noise=true , address=212.102.52.215:4376 , piece_id=25KT756B5NJCE44RWE22WJ7O5VXP7VJR6YO3M3LTRFX3CIYPVUQA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.545402781 (15h48m30.32874288s) [noise=true , address=212.102.52.215:4382 , piece_id=JCRZDFY2O6CLCTURWS7KRMWQSDNOURAA3GLPKB4P2QO3UUNB26SA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.545463316 (15h48m30.328803415s) [noise=true , address=212.102.52.214:4360 , piece_id=73PXBGVXO5KGNJDVS75DBD56COZQYSP26RBS5GND2BLG6JQLMXZA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546346461 (15h48m30.32968656s) [noise=true , address=212.102.52.216:4402 , piece_id=B5JT2RXBOTI4IUOCRZKPKA6LY6C3IGQGHWWWOXJ5FCYC64EOBIEQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546403570 (15h48m30.329743669s) [noise=true , address=212.102.52.217:4424 , piece_id=FLGYPCAZR5EF3FDQ5MT7CHBA5HDLM4CQOS6HCPVML2XKRLFGOGGQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546993276 (15h48m30.330333375s) [noise=true , address=212.102.52.215:4386 , piece_id=SJIDT4PIAIMA7QFS64FI6FMNRRO6Q5S4PVQGUKPJQBPJ7N7ZFL2A, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.548021087 (15h48m30.331361186s) [noise=true , address=212.102.52.216:4396 , piece_id=RYO7RQ26ZTRMDK22XCED2JM2K7WPEI57X25HMT5LQ73RIE7WTTYA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.548283338 (15h48m30.331623437s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.549042276 (15h48m30.332382365s) [noise=true , address=212.102.52.213:4336 , piece_id=MGI6NLZCQPDNW5I42M4NL5FWKWNOU7RFW4TZJG2LTRZATP2C742Q, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.549395759 (15h48m30.332735858s) [noise=true , address=212.102.52.217:4435 , piece_id=RNBPLJS4P5H4B24FX7XIFKRUUAWTH3PFEA3JOYAS4LCNWUUKGSBQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.550129380 (15h48m30.333469479s) [noise=true , address=212.102.52.217:4426 , piece_id=EXMYVMU7ZI4ZVELF2EMJNOZGDZAB4QZLJZ3M5DBNKIP433NFCUTA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:45.613527985 (15h48m31.396868084s) [upload=15135]: Next returned nil segment
May 25 17:52:45.613540610 (15h48m31.396880709s) [upload=15135]: Waiting for error group managing segments...
May 25 17:52:45.894196156 (15h48m31.677536255s) [noise=true , address=212.102.52.216:4394 , piece_id=4P3VUGG3TJGW6YENC6WC4J2EHG6TQNDMPXUHROT5BHRU3246ZCYA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.902564564 (15h48m31.685904663s) [noise=true , address=212.102.52.213:4336 , piece_id=MGI6NLZCQPDNW5I42M4NL5FWKWNOU7RFW4TZJG2LTRZATP2C742Q, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.902884974 (15h48m31.686225073s) [noise=true , address=212.102.52.216:4396 , piece_id=RYO7RQ26ZTRMDK22XCED2JM2K7WPEI57X25HMT5LQ73RIE7WTTYA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.903550163 (15h48m31.686890263s) [noise=true , address=212.102.52.216:4402 , piece_id=B5JT2RXBOTI4IUOCRZKPKA6LY6C3IGQGHWWWOXJ5FCYC64EOBIEQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.913910772 (15h48m31.697250881s) [noise=true , address=212.102.52.213:4333 , piece_id=TJKMUCX6JFTHDWN4WFBSLPQXI2PQFVZHNTXMOG25UYZOCTIE6LMQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.917060331 (15h48m31.70040043s) [noise=true , address=212.102.52.214:4360 , piece_id=73PXBGVXO5KGNJDVS75DBD56COZQYSP26RBS5GND2BLG6JQLMXZA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.918744335 (15h48m31.702084434s) [noise=true , address=212.102.52.215:4386 , piece_id=SJIDT4PIAIMA7QFS64FI6FMNRRO6Q5S4PVQGUKPJQBPJ7N7ZFL2A, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.924271771 (15h48m31.70761187s) [noise=true , address=212.102.52.216:4408 , piece_id=EPE6MXFOZTO5PT3SLA3U6HI6UJWMNBPWAPI7LLGPB35CRKB3AZOA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.925897152 (15h48m31.709237251s) [noise=true , address=212.102.52.215:4382 , piece_id=JCRZDFY2O6CLCTURWS7KRMWQSDNOURAA3GLPKB4P2QO3UUNB26SA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.926060985 (15h48m31.709401084s) [noise=true , address=212.102.52.217:4435 , piece_id=RNBPLJS4P5H4B24FX7XIFKRUUAWTH3PFEA3JOYAS4LCNWUUKGSBQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.927768653 (15h48m31.711108752s) [noise=true , address=212.102.52.215:4376 , piece_id=25KT756B5NJCE44RWE22WJ7O5VXP7VJR6YO3M3LTRFX3CIYPVUQA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.929649962 (15h48m31.712990061s) [noise=true , address=212.102.52.214:4364 , piece_id=MU7IJZCB752M4M7LFCEJZCDEJJY6S5M63KH4YBSIIQN5LPHW6YQA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.957068207 (15h48m31.740408307s) [noise=true , address=212.102.52.215:4380 , piece_id=2S2FDBIAGAP7AMF3CL54BECVIGDEXDDG5OHT23FJKWPSVOUPN6RA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.957729358 (15h48m31.741069457s) [noise=true , address=212.102.52.213:4324 , piece_id=YONLVC2ARQPU3ZJOSJSG2WX6SOA65DJYA5GMIEQ5ZJ5DUPR6UEFQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959196257 (15h48m31.742536356s) [noise=true , address=212.102.52.217:4426 , piece_id=EXMYVMU7ZI4ZVELF2EMJNOZGDZAB4QZLJZ3M5DBNKIP433NFCUTA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959726810 (15h48m31.743066909s) [noise=true , address=212.102.52.217:4424 , piece_id=FLGYPCAZR5EF3FDQ5MT7CHBA5HDLM4CQOS6HCPVML2XKRLFGOGGQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959735406 (15h48m31.743075505s) [seg_pos={0 0}, upload=15135]: Satisfied cohort requirements after 16 pieces
May 25 17:52:45.959899980 (15h48m31.743240079s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MbNn3rECMrMhp1uJJRHy8v3DZkKaBEcZGwPMGumvPUvvKu5gM): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959906273 (15h48m31.743246372s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.959965084 (15h48m31.743305183s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eyfVMwFVb4Q5nbc54PSLebuyWfm2rCgCfGrNc6AovSWqe6Upn): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959969824 (15h48m31.743309923s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.959978169 (15h48m31.743318269s) [noise=true , address=212.102.52.214:4356 , piece_id=UAHRGRD6YLJZOMHHBNHHONX5UZBS5W5MJETFZJZMBBP3LPJ3CP5A, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1263MPtQZy9iB1pxhBYqZ9tgqitVmjm2X7gyehcdSRja9kWfPKd): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959986315 (15h48m31.743326414s) [noise=true , address=212.102.52.214:4356 , piece_id=UAHRGRD6YLJZOMHHBNHHONX5UZBS5W5MJETFZJZMBBP3LPJ3CP5A, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.960111764 (15h48m31.743451863s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Tyfw7Bp6ma6jTKyBJaSfr5u8w2CwExJVAwkeabjMnsHWWsDdf): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.960116693 (15h48m31.743456792s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.960130370 (15h48m31.743470469s) [seg_pos={0 0}, upload=15135]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 17:52:45.960200974 (15h48m31.743541073s) [upload=15135]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 17:52:45.960205563 (15h48m31.743545662s) [upload=15135]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 17:52:48.787245030 (15h48m34.570585129s) [upload=15135]: Done uploading segments...
May 25 17:52:48.787262443 (15h48m34.570602542s) [upload=15135]: Upload finished. err: <nil>
Duration: 5.056568011s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
680.730998ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7e710747942acdb2
FAIL (2026-05-25 18:04:56 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 680.752598ms
core
satellite
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.......X..........................................X.........] duration:
4.104169633s trace:
https://jaeger.dev.storj.io/trace/14eb8131f680ed75
FAIL (2026-05-25 18:04:56 UTC)
orbiter warning: context deadline exceeded
Duration: 4.10419362s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [........X..........................................X........] duration:
3.722996801s trace:
https://jaeger.dev.storj.io/trace/14eb8131f680ed75
FAIL (2026-05-25 18:04:23 UTC)
orbiter warning: context deadline exceeded
Duration: 3.723018702s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
12 out of 60 is failed (threshold=3) [.X......X................X....X.......X......X..XX.XX.X....X] duration:
1m0.000699587s trace:
https://jaeger.dev.storj.io/trace/fac1154c0207b7
FAIL (2026-05-25 18:04:12 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000713903s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
680.207439ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7e710747942acdb2
FAIL (2026-05-25 18:04:00 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 680.226525ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
4.397162869s trace:
https://jaeger.dev.storj.io/trace/a5682f02df33c7c
FAIL (2026-05-25 18:03:55 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ZCYA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543483940 (15h48m30.326824029s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543527172 (15h48m30.326867271s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543870378 (15h48m30.327210477s) [noise=true , address=212.102.52.216:4408 , piece_id=EPE6MXFOZTO5PT3SLA3U6HI6UJWMNBPWAPI7LLGPB35CRKB3AZOA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.544411289 (15h48m30.327751388s) [noise=true , address=212.102.52.214:4364 , piece_id=MU7IJZCB752M4M7LFCEJZCDEJJY6S5M63KH4YBSIIQN5LPHW6YQA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.544595600 (15h48m30.327935699s) [noise=true , address=212.102.52.215:4376 , piece_id=25KT756B5NJCE44RWE22WJ7O5VXP7VJR6YO3M3LTRFX3CIYPVUQA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.545402781 (15h48m30.32874288s) [noise=true , address=212.102.52.215:4382 , piece_id=JCRZDFY2O6CLCTURWS7KRMWQSDNOURAA3GLPKB4P2QO3UUNB26SA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.545463316 (15h48m30.328803415s) [noise=true , address=212.102.52.214:4360 , piece_id=73PXBGVXO5KGNJDVS75DBD56COZQYSP26RBS5GND2BLG6JQLMXZA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546346461 (15h48m30.32968656s) [noise=true , address=212.102.52.216:4402 , piece_id=B5JT2RXBOTI4IUOCRZKPKA6LY6C3IGQGHWWWOXJ5FCYC64EOBIEQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546403570 (15h48m30.329743669s) [noise=true , address=212.102.52.217:4424 , piece_id=FLGYPCAZR5EF3FDQ5MT7CHBA5HDLM4CQOS6HCPVML2XKRLFGOGGQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546993276 (15h48m30.330333375s) [noise=true , address=212.102.52.215:4386 , piece_id=SJIDT4PIAIMA7QFS64FI6FMNRRO6Q5S4PVQGUKPJQBPJ7N7ZFL2A, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.548021087 (15h48m30.331361186s) [noise=true , address=212.102.52.216:4396 , piece_id=RYO7RQ26ZTRMDK22XCED2JM2K7WPEI57X25HMT5LQ73RIE7WTTYA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.548283338 (15h48m30.331623437s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.549042276 (15h48m30.332382365s) [noise=true , address=212.102.52.213:4336 , piece_id=MGI6NLZCQPDNW5I42M4NL5FWKWNOU7RFW4TZJG2LTRZATP2C742Q, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.549395759 (15h48m30.332735858s) [noise=true , address=212.102.52.217:4435 , piece_id=RNBPLJS4P5H4B24FX7XIFKRUUAWTH3PFEA3JOYAS4LCNWUUKGSBQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.550129380 (15h48m30.333469479s) [noise=true , address=212.102.52.217:4426 , piece_id=EXMYVMU7ZI4ZVELF2EMJNOZGDZAB4QZLJZ3M5DBNKIP433NFCUTA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:45.613527985 (15h48m31.396868084s) [upload=15135]: Next returned nil segment
May 25 17:52:45.613540610 (15h48m31.396880709s) [upload=15135]: Waiting for error group managing segments...
May 25 17:52:45.894196156 (15h48m31.677536255s) [noise=true , address=212.102.52.216:4394 , piece_id=4P3VUGG3TJGW6YENC6WC4J2EHG6TQNDMPXUHROT5BHRU3246ZCYA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.902564564 (15h48m31.685904663s) [noise=true , address=212.102.52.213:4336 , piece_id=MGI6NLZCQPDNW5I42M4NL5FWKWNOU7RFW4TZJG2LTRZATP2C742Q, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.902884974 (15h48m31.686225073s) [noise=true , address=212.102.52.216:4396 , piece_id=RYO7RQ26ZTRMDK22XCED2JM2K7WPEI57X25HMT5LQ73RIE7WTTYA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.903550163 (15h48m31.686890263s) [noise=true , address=212.102.52.216:4402 , piece_id=B5JT2RXBOTI4IUOCRZKPKA6LY6C3IGQGHWWWOXJ5FCYC64EOBIEQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.913910772 (15h48m31.697250881s) [noise=true , address=212.102.52.213:4333 , piece_id=TJKMUCX6JFTHDWN4WFBSLPQXI2PQFVZHNTXMOG25UYZOCTIE6LMQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.917060331 (15h48m31.70040043s) [noise=true , address=212.102.52.214:4360 , piece_id=73PXBGVXO5KGNJDVS75DBD56COZQYSP26RBS5GND2BLG6JQLMXZA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.918744335 (15h48m31.702084434s) [noise=true , address=212.102.52.215:4386 , piece_id=SJIDT4PIAIMA7QFS64FI6FMNRRO6Q5S4PVQGUKPJQBPJ7N7ZFL2A, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.924271771 (15h48m31.70761187s) [noise=true , address=212.102.52.216:4408 , piece_id=EPE6MXFOZTO5PT3SLA3U6HI6UJWMNBPWAPI7LLGPB35CRKB3AZOA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.925897152 (15h48m31.709237251s) [noise=true , address=212.102.52.215:4382 , piece_id=JCRZDFY2O6CLCTURWS7KRMWQSDNOURAA3GLPKB4P2QO3UUNB26SA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.926060985 (15h48m31.709401084s) [noise=true , address=212.102.52.217:4435 , piece_id=RNBPLJS4P5H4B24FX7XIFKRUUAWTH3PFEA3JOYAS4LCNWUUKGSBQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.927768653 (15h48m31.711108752s) [noise=true , address=212.102.52.215:4376 , piece_id=25KT756B5NJCE44RWE22WJ7O5VXP7VJR6YO3M3LTRFX3CIYPVUQA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.929649962 (15h48m31.712990061s) [noise=true , address=212.102.52.214:4364 , piece_id=MU7IJZCB752M4M7LFCEJZCDEJJY6S5M63KH4YBSIIQN5LPHW6YQA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.957068207 (15h48m31.740408307s) [noise=true , address=212.102.52.215:4380 , piece_id=2S2FDBIAGAP7AMF3CL54BECVIGDEXDDG5OHT23FJKWPSVOUPN6RA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.957729358 (15h48m31.741069457s) [noise=true , address=212.102.52.213:4324 , piece_id=YONLVC2ARQPU3ZJOSJSG2WX6SOA65DJYA5GMIEQ5ZJ5DUPR6UEFQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959196257 (15h48m31.742536356s) [noise=true , address=212.102.52.217:4426 , piece_id=EXMYVMU7ZI4ZVELF2EMJNOZGDZAB4QZLJZ3M5DBNKIP433NFCUTA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959726810 (15h48m31.743066909s) [noise=true , address=212.102.52.217:4424 , piece_id=FLGYPCAZR5EF3FDQ5MT7CHBA5HDLM4CQOS6HCPVML2XKRLFGOGGQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959735406 (15h48m31.743075505s) [seg_pos={0 0}, upload=15135]: Satisfied cohort requirements after 16 pieces
May 25 17:52:45.959899980 (15h48m31.743240079s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MbNn3rECMrMhp1uJJRHy8v3DZkKaBEcZGwPMGumvPUvvKu5gM): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959906273 (15h48m31.743246372s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.959965084 (15h48m31.743305183s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eyfVMwFVb4Q5nbc54PSLebuyWfm2rCgCfGrNc6AovSWqe6Upn): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959969824 (15h48m31.743309923s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.959978169 (15h48m31.743318269s) [noise=true , address=212.102.52.214:4356 , piece_id=UAHRGRD6YLJZOMHHBNHHONX5UZBS5W5MJETFZJZMBBP3LPJ3CP5A, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1263MPtQZy9iB1pxhBYqZ9tgqitVmjm2X7gyehcdSRja9kWfPKd): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959986315 (15h48m31.743326414s) [noise=true , address=212.102.52.214:4356 , piece_id=UAHRGRD6YLJZOMHHBNHHONX5UZBS5W5MJETFZJZMBBP3LPJ3CP5A, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.960111764 (15h48m31.743451863s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Tyfw7Bp6ma6jTKyBJaSfr5u8w2CwExJVAwkeabjMnsHWWsDdf): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.960116693 (15h48m31.743456792s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.960130370 (15h48m31.743470469s) [seg_pos={0 0}, upload=15135]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 17:52:45.960200974 (15h48m31.743541073s) [upload=15135]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 17:52:45.960205563 (15h48m31.743545662s) [upload=15135]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 17:52:48.787245030 (15h48m34.570585129s) [upload=15135]: Done uploading segments...
May 25 17:52:48.787262443 (15h48m34.570602542s) [upload=15135]: Upload finished. err: <nil>
Duration: 4.397188409s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.........X..........................................X.......] duration:
3.949872009s trace:
https://jaeger.dev.storj.io/trace/14eb8131f680ed75
FAIL (2026-05-25 18:03:54 UTC)
orbiter warning: context deadline exceeded
Duration: 3.950272083s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
3.92624891s trace:
https://jaeger.dev.storj.io/trace/a5682f02df33c7c
FAIL (2026-05-25 18:03:26 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ZCYA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543483940 (15h48m30.326824029s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543527172 (15h48m30.326867271s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543870378 (15h48m30.327210477s) [noise=true , address=212.102.52.216:4408 , piece_id=EPE6MXFOZTO5PT3SLA3U6HI6UJWMNBPWAPI7LLGPB35CRKB3AZOA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.544411289 (15h48m30.327751388s) [noise=true , address=212.102.52.214:4364 , piece_id=MU7IJZCB752M4M7LFCEJZCDEJJY6S5M63KH4YBSIIQN5LPHW6YQA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.544595600 (15h48m30.327935699s) [noise=true , address=212.102.52.215:4376 , piece_id=25KT756B5NJCE44RWE22WJ7O5VXP7VJR6YO3M3LTRFX3CIYPVUQA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.545402781 (15h48m30.32874288s) [noise=true , address=212.102.52.215:4382 , piece_id=JCRZDFY2O6CLCTURWS7KRMWQSDNOURAA3GLPKB4P2QO3UUNB26SA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.545463316 (15h48m30.328803415s) [noise=true , address=212.102.52.214:4360 , piece_id=73PXBGVXO5KGNJDVS75DBD56COZQYSP26RBS5GND2BLG6JQLMXZA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546346461 (15h48m30.32968656s) [noise=true , address=212.102.52.216:4402 , piece_id=B5JT2RXBOTI4IUOCRZKPKA6LY6C3IGQGHWWWOXJ5FCYC64EOBIEQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546403570 (15h48m30.329743669s) [noise=true , address=212.102.52.217:4424 , piece_id=FLGYPCAZR5EF3FDQ5MT7CHBA5HDLM4CQOS6HCPVML2XKRLFGOGGQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546993276 (15h48m30.330333375s) [noise=true , address=212.102.52.215:4386 , piece_id=SJIDT4PIAIMA7QFS64FI6FMNRRO6Q5S4PVQGUKPJQBPJ7N7ZFL2A, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.548021087 (15h48m30.331361186s) [noise=true , address=212.102.52.216:4396 , piece_id=RYO7RQ26ZTRMDK22XCED2JM2K7WPEI57X25HMT5LQ73RIE7WTTYA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.548283338 (15h48m30.331623437s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.549042276 (15h48m30.332382365s) [noise=true , address=212.102.52.213:4336 , piece_id=MGI6NLZCQPDNW5I42M4NL5FWKWNOU7RFW4TZJG2LTRZATP2C742Q, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.549395759 (15h48m30.332735858s) [noise=true , address=212.102.52.217:4435 , piece_id=RNBPLJS4P5H4B24FX7XIFKRUUAWTH3PFEA3JOYAS4LCNWUUKGSBQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.550129380 (15h48m30.333469479s) [noise=true , address=212.102.52.217:4426 , piece_id=EXMYVMU7ZI4ZVELF2EMJNOZGDZAB4QZLJZ3M5DBNKIP433NFCUTA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:45.613527985 (15h48m31.396868084s) [upload=15135]: Next returned nil segment
May 25 17:52:45.613540610 (15h48m31.396880709s) [upload=15135]: Waiting for error group managing segments...
May 25 17:52:45.894196156 (15h48m31.677536255s) [noise=true , address=212.102.52.216:4394 , piece_id=4P3VUGG3TJGW6YENC6WC4J2EHG6TQNDMPXUHROT5BHRU3246ZCYA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.902564564 (15h48m31.685904663s) [noise=true , address=212.102.52.213:4336 , piece_id=MGI6NLZCQPDNW5I42M4NL5FWKWNOU7RFW4TZJG2LTRZATP2C742Q, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.902884974 (15h48m31.686225073s) [noise=true , address=212.102.52.216:4396 , piece_id=RYO7RQ26ZTRMDK22XCED2JM2K7WPEI57X25HMT5LQ73RIE7WTTYA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.903550163 (15h48m31.686890263s) [noise=true , address=212.102.52.216:4402 , piece_id=B5JT2RXBOTI4IUOCRZKPKA6LY6C3IGQGHWWWOXJ5FCYC64EOBIEQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.913910772 (15h48m31.697250881s) [noise=true , address=212.102.52.213:4333 , piece_id=TJKMUCX6JFTHDWN4WFBSLPQXI2PQFVZHNTXMOG25UYZOCTIE6LMQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.917060331 (15h48m31.70040043s) [noise=true , address=212.102.52.214:4360 , piece_id=73PXBGVXO5KGNJDVS75DBD56COZQYSP26RBS5GND2BLG6JQLMXZA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.918744335 (15h48m31.702084434s) [noise=true , address=212.102.52.215:4386 , piece_id=SJIDT4PIAIMA7QFS64FI6FMNRRO6Q5S4PVQGUKPJQBPJ7N7ZFL2A, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.924271771 (15h48m31.70761187s) [noise=true , address=212.102.52.216:4408 , piece_id=EPE6MXFOZTO5PT3SLA3U6HI6UJWMNBPWAPI7LLGPB35CRKB3AZOA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.925897152 (15h48m31.709237251s) [noise=true , address=212.102.52.215:4382 , piece_id=JCRZDFY2O6CLCTURWS7KRMWQSDNOURAA3GLPKB4P2QO3UUNB26SA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.926060985 (15h48m31.709401084s) [noise=true , address=212.102.52.217:4435 , piece_id=RNBPLJS4P5H4B24FX7XIFKRUUAWTH3PFEA3JOYAS4LCNWUUKGSBQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.927768653 (15h48m31.711108752s) [noise=true , address=212.102.52.215:4376 , piece_id=25KT756B5NJCE44RWE22WJ7O5VXP7VJR6YO3M3LTRFX3CIYPVUQA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.929649962 (15h48m31.712990061s) [noise=true , address=212.102.52.214:4364 , piece_id=MU7IJZCB752M4M7LFCEJZCDEJJY6S5M63KH4YBSIIQN5LPHW6YQA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.957068207 (15h48m31.740408307s) [noise=true , address=212.102.52.215:4380 , piece_id=2S2FDBIAGAP7AMF3CL54BECVIGDEXDDG5OHT23FJKWPSVOUPN6RA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.957729358 (15h48m31.741069457s) [noise=true , address=212.102.52.213:4324 , piece_id=YONLVC2ARQPU3ZJOSJSG2WX6SOA65DJYA5GMIEQ5ZJ5DUPR6UEFQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959196257 (15h48m31.742536356s) [noise=true , address=212.102.52.217:4426 , piece_id=EXMYVMU7ZI4ZVELF2EMJNOZGDZAB4QZLJZ3M5DBNKIP433NFCUTA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959726810 (15h48m31.743066909s) [noise=true , address=212.102.52.217:4424 , piece_id=FLGYPCAZR5EF3FDQ5MT7CHBA5HDLM4CQOS6HCPVML2XKRLFGOGGQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959735406 (15h48m31.743075505s) [seg_pos={0 0}, upload=15135]: Satisfied cohort requirements after 16 pieces
May 25 17:52:45.959899980 (15h48m31.743240079s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MbNn3rECMrMhp1uJJRHy8v3DZkKaBEcZGwPMGumvPUvvKu5gM): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959906273 (15h48m31.743246372s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.959965084 (15h48m31.743305183s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eyfVMwFVb4Q5nbc54PSLebuyWfm2rCgCfGrNc6AovSWqe6Upn): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959969824 (15h48m31.743309923s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.959978169 (15h48m31.743318269s) [noise=true , address=212.102.52.214:4356 , piece_id=UAHRGRD6YLJZOMHHBNHHONX5UZBS5W5MJETFZJZMBBP3LPJ3CP5A, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1263MPtQZy9iB1pxhBYqZ9tgqitVmjm2X7gyehcdSRja9kWfPKd): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959986315 (15h48m31.743326414s) [noise=true , address=212.102.52.214:4356 , piece_id=UAHRGRD6YLJZOMHHBNHHONX5UZBS5W5MJETFZJZMBBP3LPJ3CP5A, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.960111764 (15h48m31.743451863s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Tyfw7Bp6ma6jTKyBJaSfr5u8w2CwExJVAwkeabjMnsHWWsDdf): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.960116693 (15h48m31.743456792s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.960130370 (15h48m31.743470469s) [seg_pos={0 0}, upload=15135]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 17:52:45.960200974 (15h48m31.743541073s) [upload=15135]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 17:52:45.960205563 (15h48m31.743545662s) [upload=15135]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 17:52:48.787245030 (15h48m34.570585129s) [upload=15135]: Done uploading segments...
May 25 17:52:48.787262443 (15h48m34.570602542s) [upload=15135]: Upload finished. err: <nil>
Duration: 3.926266104s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
677.103055ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7e710747942acdb2
FAIL (2026-05-25 18:03:20 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 677.123635ms
core
satellite
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [..........X..........................................X......] duration:
3.783689884s trace:
https://jaeger.dev.storj.io/trace/14eb8131f680ed75
FAIL (2026-05-25 18:02:57 UTC)
orbiter warning: context deadline exceeded
Duration: 3.783727286s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
673.278329ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7e710747942acdb2
FAIL (2026-05-25 18:02:55 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 673.301774ms
core
satellite
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
11 out of 60 is failed (threshold=3) [..X......X................X....X.......X......X..XX.XX.X....] duration:
13.036857524s trace:
https://jaeger.dev.storj.io/trace/118234d3333e412c
FAIL (2026-05-25 18:02:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 13.03690188s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
4.084236533s trace:
https://jaeger.dev.storj.io/trace/a5682f02df33c7c
FAIL (2026-05-25 18:02:45 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ZCYA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543483940 (15h48m30.326824029s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543527172 (15h48m30.326867271s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543870378 (15h48m30.327210477s) [noise=true , address=212.102.52.216:4408 , piece_id=EPE6MXFOZTO5PT3SLA3U6HI6UJWMNBPWAPI7LLGPB35CRKB3AZOA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.544411289 (15h48m30.327751388s) [noise=true , address=212.102.52.214:4364 , piece_id=MU7IJZCB752M4M7LFCEJZCDEJJY6S5M63KH4YBSIIQN5LPHW6YQA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.544595600 (15h48m30.327935699s) [noise=true , address=212.102.52.215:4376 , piece_id=25KT756B5NJCE44RWE22WJ7O5VXP7VJR6YO3M3LTRFX3CIYPVUQA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.545402781 (15h48m30.32874288s) [noise=true , address=212.102.52.215:4382 , piece_id=JCRZDFY2O6CLCTURWS7KRMWQSDNOURAA3GLPKB4P2QO3UUNB26SA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.545463316 (15h48m30.328803415s) [noise=true , address=212.102.52.214:4360 , piece_id=73PXBGVXO5KGNJDVS75DBD56COZQYSP26RBS5GND2BLG6JQLMXZA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546346461 (15h48m30.32968656s) [noise=true , address=212.102.52.216:4402 , piece_id=B5JT2RXBOTI4IUOCRZKPKA6LY6C3IGQGHWWWOXJ5FCYC64EOBIEQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546403570 (15h48m30.329743669s) [noise=true , address=212.102.52.217:4424 , piece_id=FLGYPCAZR5EF3FDQ5MT7CHBA5HDLM4CQOS6HCPVML2XKRLFGOGGQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546993276 (15h48m30.330333375s) [noise=true , address=212.102.52.215:4386 , piece_id=SJIDT4PIAIMA7QFS64FI6FMNRRO6Q5S4PVQGUKPJQBPJ7N7ZFL2A, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.548021087 (15h48m30.331361186s) [noise=true , address=212.102.52.216:4396 , piece_id=RYO7RQ26ZTRMDK22XCED2JM2K7WPEI57X25HMT5LQ73RIE7WTTYA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.548283338 (15h48m30.331623437s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.549042276 (15h48m30.332382365s) [noise=true , address=212.102.52.213:4336 , piece_id=MGI6NLZCQPDNW5I42M4NL5FWKWNOU7RFW4TZJG2LTRZATP2C742Q, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.549395759 (15h48m30.332735858s) [noise=true , address=212.102.52.217:4435 , piece_id=RNBPLJS4P5H4B24FX7XIFKRUUAWTH3PFEA3JOYAS4LCNWUUKGSBQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.550129380 (15h48m30.333469479s) [noise=true , address=212.102.52.217:4426 , piece_id=EXMYVMU7ZI4ZVELF2EMJNOZGDZAB4QZLJZ3M5DBNKIP433NFCUTA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:45.613527985 (15h48m31.396868084s) [upload=15135]: Next returned nil segment
May 25 17:52:45.613540610 (15h48m31.396880709s) [upload=15135]: Waiting for error group managing segments...
May 25 17:52:45.894196156 (15h48m31.677536255s) [noise=true , address=212.102.52.216:4394 , piece_id=4P3VUGG3TJGW6YENC6WC4J2EHG6TQNDMPXUHROT5BHRU3246ZCYA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.902564564 (15h48m31.685904663s) [noise=true , address=212.102.52.213:4336 , piece_id=MGI6NLZCQPDNW5I42M4NL5FWKWNOU7RFW4TZJG2LTRZATP2C742Q, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.902884974 (15h48m31.686225073s) [noise=true , address=212.102.52.216:4396 , piece_id=RYO7RQ26ZTRMDK22XCED2JM2K7WPEI57X25HMT5LQ73RIE7WTTYA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.903550163 (15h48m31.686890263s) [noise=true , address=212.102.52.216:4402 , piece_id=B5JT2RXBOTI4IUOCRZKPKA6LY6C3IGQGHWWWOXJ5FCYC64EOBIEQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.913910772 (15h48m31.697250881s) [noise=true , address=212.102.52.213:4333 , piece_id=TJKMUCX6JFTHDWN4WFBSLPQXI2PQFVZHNTXMOG25UYZOCTIE6LMQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.917060331 (15h48m31.70040043s) [noise=true , address=212.102.52.214:4360 , piece_id=73PXBGVXO5KGNJDVS75DBD56COZQYSP26RBS5GND2BLG6JQLMXZA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.918744335 (15h48m31.702084434s) [noise=true , address=212.102.52.215:4386 , piece_id=SJIDT4PIAIMA7QFS64FI6FMNRRO6Q5S4PVQGUKPJQBPJ7N7ZFL2A, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.924271771 (15h48m31.70761187s) [noise=true , address=212.102.52.216:4408 , piece_id=EPE6MXFOZTO5PT3SLA3U6HI6UJWMNBPWAPI7LLGPB35CRKB3AZOA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.925897152 (15h48m31.709237251s) [noise=true , address=212.102.52.215:4382 , piece_id=JCRZDFY2O6CLCTURWS7KRMWQSDNOURAA3GLPKB4P2QO3UUNB26SA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.926060985 (15h48m31.709401084s) [noise=true , address=212.102.52.217:4435 , piece_id=RNBPLJS4P5H4B24FX7XIFKRUUAWTH3PFEA3JOYAS4LCNWUUKGSBQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.927768653 (15h48m31.711108752s) [noise=true , address=212.102.52.215:4376 , piece_id=25KT756B5NJCE44RWE22WJ7O5VXP7VJR6YO3M3LTRFX3CIYPVUQA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.929649962 (15h48m31.712990061s) [noise=true , address=212.102.52.214:4364 , piece_id=MU7IJZCB752M4M7LFCEJZCDEJJY6S5M63KH4YBSIIQN5LPHW6YQA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.957068207 (15h48m31.740408307s) [noise=true , address=212.102.52.215:4380 , piece_id=2S2FDBIAGAP7AMF3CL54BECVIGDEXDDG5OHT23FJKWPSVOUPN6RA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.957729358 (15h48m31.741069457s) [noise=true , address=212.102.52.213:4324 , piece_id=YONLVC2ARQPU3ZJOSJSG2WX6SOA65DJYA5GMIEQ5ZJ5DUPR6UEFQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959196257 (15h48m31.742536356s) [noise=true , address=212.102.52.217:4426 , piece_id=EXMYVMU7ZI4ZVELF2EMJNOZGDZAB4QZLJZ3M5DBNKIP433NFCUTA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959726810 (15h48m31.743066909s) [noise=true , address=212.102.52.217:4424 , piece_id=FLGYPCAZR5EF3FDQ5MT7CHBA5HDLM4CQOS6HCPVML2XKRLFGOGGQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959735406 (15h48m31.743075505s) [seg_pos={0 0}, upload=15135]: Satisfied cohort requirements after 16 pieces
May 25 17:52:45.959899980 (15h48m31.743240079s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MbNn3rECMrMhp1uJJRHy8v3DZkKaBEcZGwPMGumvPUvvKu5gM): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959906273 (15h48m31.743246372s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.959965084 (15h48m31.743305183s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eyfVMwFVb4Q5nbc54PSLebuyWfm2rCgCfGrNc6AovSWqe6Upn): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959969824 (15h48m31.743309923s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.959978169 (15h48m31.743318269s) [noise=true , address=212.102.52.214:4356 , piece_id=UAHRGRD6YLJZOMHHBNHHONX5UZBS5W5MJETFZJZMBBP3LPJ3CP5A, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1263MPtQZy9iB1pxhBYqZ9tgqitVmjm2X7gyehcdSRja9kWfPKd): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959986315 (15h48m31.743326414s) [noise=true , address=212.102.52.214:4356 , piece_id=UAHRGRD6YLJZOMHHBNHHONX5UZBS5W5MJETFZJZMBBP3LPJ3CP5A, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.960111764 (15h48m31.743451863s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Tyfw7Bp6ma6jTKyBJaSfr5u8w2CwExJVAwkeabjMnsHWWsDdf): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.960116693 (15h48m31.743456792s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.960130370 (15h48m31.743470469s) [seg_pos={0 0}, upload=15135]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 17:52:45.960200974 (15h48m31.743541073s) [upload=15135]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 17:52:45.960205563 (15h48m31.743545662s) [upload=15135]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 17:52:48.787245030 (15h48m34.570585129s) [upload=15135]: Done uploading segments...
May 25 17:52:48.787262443 (15h48m34.570602542s) [upload=15135]: Upload finished. err: <nil>
Duration: 4.084261372s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
678.872982ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7e710747942acdb2
FAIL (2026-05-25 18:02:37 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 678.91412ms
core
satellite
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
11 out of 60 is failed (threshold=3) [...X......X................X....X.......X......X..XX.XX.X...] duration:
11.560272095s trace:
https://jaeger.dev.storj.io/trace/118234d3333e412c
FAIL (2026-05-25 18:02:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 11.560349623s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
11 out of 60 is failed (threshold=3) [....X......X................X....X.......X......X..XX.XX.X..] duration:
21.499436974s trace:
https://jaeger.dev.storj.io/trace/118234d3333e412c
FAIL (2026-05-25 18:02:05 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 21.499445932s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
680.03582ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7e710747942acdb2
FAIL (2026-05-25 18:01:54 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 680.057301ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
4.1708538s trace:
https://jaeger.dev.storj.io/trace/a5682f02df33c7c
FAIL (2026-05-25 18:01:44 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ZCYA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543483940 (15h48m30.326824029s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543527172 (15h48m30.326867271s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543870378 (15h48m30.327210477s) [noise=true , address=212.102.52.216:4408 , piece_id=EPE6MXFOZTO5PT3SLA3U6HI6UJWMNBPWAPI7LLGPB35CRKB3AZOA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.544411289 (15h48m30.327751388s) [noise=true , address=212.102.52.214:4364 , piece_id=MU7IJZCB752M4M7LFCEJZCDEJJY6S5M63KH4YBSIIQN5LPHW6YQA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.544595600 (15h48m30.327935699s) [noise=true , address=212.102.52.215:4376 , piece_id=25KT756B5NJCE44RWE22WJ7O5VXP7VJR6YO3M3LTRFX3CIYPVUQA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.545402781 (15h48m30.32874288s) [noise=true , address=212.102.52.215:4382 , piece_id=JCRZDFY2O6CLCTURWS7KRMWQSDNOURAA3GLPKB4P2QO3UUNB26SA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.545463316 (15h48m30.328803415s) [noise=true , address=212.102.52.214:4360 , piece_id=73PXBGVXO5KGNJDVS75DBD56COZQYSP26RBS5GND2BLG6JQLMXZA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546346461 (15h48m30.32968656s) [noise=true , address=212.102.52.216:4402 , piece_id=B5JT2RXBOTI4IUOCRZKPKA6LY6C3IGQGHWWWOXJ5FCYC64EOBIEQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546403570 (15h48m30.329743669s) [noise=true , address=212.102.52.217:4424 , piece_id=FLGYPCAZR5EF3FDQ5MT7CHBA5HDLM4CQOS6HCPVML2XKRLFGOGGQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546993276 (15h48m30.330333375s) [noise=true , address=212.102.52.215:4386 , piece_id=SJIDT4PIAIMA7QFS64FI6FMNRRO6Q5S4PVQGUKPJQBPJ7N7ZFL2A, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.548021087 (15h48m30.331361186s) [noise=true , address=212.102.52.216:4396 , piece_id=RYO7RQ26ZTRMDK22XCED2JM2K7WPEI57X25HMT5LQ73RIE7WTTYA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.548283338 (15h48m30.331623437s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.549042276 (15h48m30.332382365s) [noise=true , address=212.102.52.213:4336 , piece_id=MGI6NLZCQPDNW5I42M4NL5FWKWNOU7RFW4TZJG2LTRZATP2C742Q, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.549395759 (15h48m30.332735858s) [noise=true , address=212.102.52.217:4435 , piece_id=RNBPLJS4P5H4B24FX7XIFKRUUAWTH3PFEA3JOYAS4LCNWUUKGSBQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.550129380 (15h48m30.333469479s) [noise=true , address=212.102.52.217:4426 , piece_id=EXMYVMU7ZI4ZVELF2EMJNOZGDZAB4QZLJZ3M5DBNKIP433NFCUTA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:45.613527985 (15h48m31.396868084s) [upload=15135]: Next returned nil segment
May 25 17:52:45.613540610 (15h48m31.396880709s) [upload=15135]: Waiting for error group managing segments...
May 25 17:52:45.894196156 (15h48m31.677536255s) [noise=true , address=212.102.52.216:4394 , piece_id=4P3VUGG3TJGW6YENC6WC4J2EHG6TQNDMPXUHROT5BHRU3246ZCYA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.902564564 (15h48m31.685904663s) [noise=true , address=212.102.52.213:4336 , piece_id=MGI6NLZCQPDNW5I42M4NL5FWKWNOU7RFW4TZJG2LTRZATP2C742Q, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.902884974 (15h48m31.686225073s) [noise=true , address=212.102.52.216:4396 , piece_id=RYO7RQ26ZTRMDK22XCED2JM2K7WPEI57X25HMT5LQ73RIE7WTTYA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.903550163 (15h48m31.686890263s) [noise=true , address=212.102.52.216:4402 , piece_id=B5JT2RXBOTI4IUOCRZKPKA6LY6C3IGQGHWWWOXJ5FCYC64EOBIEQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.913910772 (15h48m31.697250881s) [noise=true , address=212.102.52.213:4333 , piece_id=TJKMUCX6JFTHDWN4WFBSLPQXI2PQFVZHNTXMOG25UYZOCTIE6LMQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.917060331 (15h48m31.70040043s) [noise=true , address=212.102.52.214:4360 , piece_id=73PXBGVXO5KGNJDVS75DBD56COZQYSP26RBS5GND2BLG6JQLMXZA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.918744335 (15h48m31.702084434s) [noise=true , address=212.102.52.215:4386 , piece_id=SJIDT4PIAIMA7QFS64FI6FMNRRO6Q5S4PVQGUKPJQBPJ7N7ZFL2A, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.924271771 (15h48m31.70761187s) [noise=true , address=212.102.52.216:4408 , piece_id=EPE6MXFOZTO5PT3SLA3U6HI6UJWMNBPWAPI7LLGPB35CRKB3AZOA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.925897152 (15h48m31.709237251s) [noise=true , address=212.102.52.215:4382 , piece_id=JCRZDFY2O6CLCTURWS7KRMWQSDNOURAA3GLPKB4P2QO3UUNB26SA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.926060985 (15h48m31.709401084s) [noise=true , address=212.102.52.217:4435 , piece_id=RNBPLJS4P5H4B24FX7XIFKRUUAWTH3PFEA3JOYAS4LCNWUUKGSBQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.927768653 (15h48m31.711108752s) [noise=true , address=212.102.52.215:4376 , piece_id=25KT756B5NJCE44RWE22WJ7O5VXP7VJR6YO3M3LTRFX3CIYPVUQA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.929649962 (15h48m31.712990061s) [noise=true , address=212.102.52.214:4364 , piece_id=MU7IJZCB752M4M7LFCEJZCDEJJY6S5M63KH4YBSIIQN5LPHW6YQA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.957068207 (15h48m31.740408307s) [noise=true , address=212.102.52.215:4380 , piece_id=2S2FDBIAGAP7AMF3CL54BECVIGDEXDDG5OHT23FJKWPSVOUPN6RA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.957729358 (15h48m31.741069457s) [noise=true , address=212.102.52.213:4324 , piece_id=YONLVC2ARQPU3ZJOSJSG2WX6SOA65DJYA5GMIEQ5ZJ5DUPR6UEFQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959196257 (15h48m31.742536356s) [noise=true , address=212.102.52.217:4426 , piece_id=EXMYVMU7ZI4ZVELF2EMJNOZGDZAB4QZLJZ3M5DBNKIP433NFCUTA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959726810 (15h48m31.743066909s) [noise=true , address=212.102.52.217:4424 , piece_id=FLGYPCAZR5EF3FDQ5MT7CHBA5HDLM4CQOS6HCPVML2XKRLFGOGGQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959735406 (15h48m31.743075505s) [seg_pos={0 0}, upload=15135]: Satisfied cohort requirements after 16 pieces
May 25 17:52:45.959899980 (15h48m31.743240079s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MbNn3rECMrMhp1uJJRHy8v3DZkKaBEcZGwPMGumvPUvvKu5gM): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959906273 (15h48m31.743246372s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.959965084 (15h48m31.743305183s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eyfVMwFVb4Q5nbc54PSLebuyWfm2rCgCfGrNc6AovSWqe6Upn): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959969824 (15h48m31.743309923s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.959978169 (15h48m31.743318269s) [noise=true , address=212.102.52.214:4356 , piece_id=UAHRGRD6YLJZOMHHBNHHONX5UZBS5W5MJETFZJZMBBP3LPJ3CP5A, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1263MPtQZy9iB1pxhBYqZ9tgqitVmjm2X7gyehcdSRja9kWfPKd): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959986315 (15h48m31.743326414s) [noise=true , address=212.102.52.214:4356 , piece_id=UAHRGRD6YLJZOMHHBNHHONX5UZBS5W5MJETFZJZMBBP3LPJ3CP5A, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.960111764 (15h48m31.743451863s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Tyfw7Bp6ma6jTKyBJaSfr5u8w2CwExJVAwkeabjMnsHWWsDdf): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.960116693 (15h48m31.743456792s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.960130370 (15h48m31.743470469s) [seg_pos={0 0}, upload=15135]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 17:52:45.960200974 (15h48m31.743541073s) [upload=15135]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 17:52:45.960205563 (15h48m31.743545662s) [upload=15135]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 17:52:48.787245030 (15h48m34.570585129s) [upload=15135]: Done uploading segments...
May 25 17:52:48.787262443 (15h48m34.570602542s) [upload=15135]: Upload finished. err: <nil>
Duration: 4.170879167s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [...........X..........................................X.....] duration:
3.699385568s trace:
https://jaeger.dev.storj.io/trace/14eb8131f680ed75
FAIL (2026-05-25 18:01:43 UTC)
orbiter warning: context deadline exceeded
Duration: 3.699404473s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
11 out of 60 is failed (threshold=3) [.....X......X................X....X.......X......X..XX.XX.X.] duration:
6.681248695s trace:
https://jaeger.dev.storj.io/trace/118234d3333e412c
FAIL (2026-05-25 18:01:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.681270706s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
678.118804ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7e710747942acdb2
FAIL (2026-05-25 18:01:21 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 678.146096ms
core
satellite
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
11 out of 60 is failed (threshold=3) [......X......X................X....X.......X......X..XX.XX.X] duration:
1m0.000700731s trace:
https://jaeger.dev.storj.io/trace/118234d3333e412c
FAIL (2026-05-25 18:01:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000710059s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [............X..........................................X....] duration:
3.801417866s trace:
https://jaeger.dev.storj.io/trace/14eb8131f680ed75
FAIL (2026-05-25 18:01:18 UTC)
orbiter warning: context deadline exceeded
Duration: 3.801433887s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
4.100335257s trace:
https://jaeger.dev.storj.io/trace/a5682f02df33c7c
FAIL (2026-05-25 18:01:05 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ZCYA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543483940 (15h48m30.326824029s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543527172 (15h48m30.326867271s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543870378 (15h48m30.327210477s) [noise=true , address=212.102.52.216:4408 , piece_id=EPE6MXFOZTO5PT3SLA3U6HI6UJWMNBPWAPI7LLGPB35CRKB3AZOA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.544411289 (15h48m30.327751388s) [noise=true , address=212.102.52.214:4364 , piece_id=MU7IJZCB752M4M7LFCEJZCDEJJY6S5M63KH4YBSIIQN5LPHW6YQA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.544595600 (15h48m30.327935699s) [noise=true , address=212.102.52.215:4376 , piece_id=25KT756B5NJCE44RWE22WJ7O5VXP7VJR6YO3M3LTRFX3CIYPVUQA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.545402781 (15h48m30.32874288s) [noise=true , address=212.102.52.215:4382 , piece_id=JCRZDFY2O6CLCTURWS7KRMWQSDNOURAA3GLPKB4P2QO3UUNB26SA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.545463316 (15h48m30.328803415s) [noise=true , address=212.102.52.214:4360 , piece_id=73PXBGVXO5KGNJDVS75DBD56COZQYSP26RBS5GND2BLG6JQLMXZA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546346461 (15h48m30.32968656s) [noise=true , address=212.102.52.216:4402 , piece_id=B5JT2RXBOTI4IUOCRZKPKA6LY6C3IGQGHWWWOXJ5FCYC64EOBIEQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546403570 (15h48m30.329743669s) [noise=true , address=212.102.52.217:4424 , piece_id=FLGYPCAZR5EF3FDQ5MT7CHBA5HDLM4CQOS6HCPVML2XKRLFGOGGQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546993276 (15h48m30.330333375s) [noise=true , address=212.102.52.215:4386 , piece_id=SJIDT4PIAIMA7QFS64FI6FMNRRO6Q5S4PVQGUKPJQBPJ7N7ZFL2A, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.548021087 (15h48m30.331361186s) [noise=true , address=212.102.52.216:4396 , piece_id=RYO7RQ26ZTRMDK22XCED2JM2K7WPEI57X25HMT5LQ73RIE7WTTYA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.548283338 (15h48m30.331623437s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.549042276 (15h48m30.332382365s) [noise=true , address=212.102.52.213:4336 , piece_id=MGI6NLZCQPDNW5I42M4NL5FWKWNOU7RFW4TZJG2LTRZATP2C742Q, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.549395759 (15h48m30.332735858s) [noise=true , address=212.102.52.217:4435 , piece_id=RNBPLJS4P5H4B24FX7XIFKRUUAWTH3PFEA3JOYAS4LCNWUUKGSBQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.550129380 (15h48m30.333469479s) [noise=true , address=212.102.52.217:4426 , piece_id=EXMYVMU7ZI4ZVELF2EMJNOZGDZAB4QZLJZ3M5DBNKIP433NFCUTA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:45.613527985 (15h48m31.396868084s) [upload=15135]: Next returned nil segment
May 25 17:52:45.613540610 (15h48m31.396880709s) [upload=15135]: Waiting for error group managing segments...
May 25 17:52:45.894196156 (15h48m31.677536255s) [noise=true , address=212.102.52.216:4394 , piece_id=4P3VUGG3TJGW6YENC6WC4J2EHG6TQNDMPXUHROT5BHRU3246ZCYA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.902564564 (15h48m31.685904663s) [noise=true , address=212.102.52.213:4336 , piece_id=MGI6NLZCQPDNW5I42M4NL5FWKWNOU7RFW4TZJG2LTRZATP2C742Q, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.902884974 (15h48m31.686225073s) [noise=true , address=212.102.52.216:4396 , piece_id=RYO7RQ26ZTRMDK22XCED2JM2K7WPEI57X25HMT5LQ73RIE7WTTYA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.903550163 (15h48m31.686890263s) [noise=true , address=212.102.52.216:4402 , piece_id=B5JT2RXBOTI4IUOCRZKPKA6LY6C3IGQGHWWWOXJ5FCYC64EOBIEQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.913910772 (15h48m31.697250881s) [noise=true , address=212.102.52.213:4333 , piece_id=TJKMUCX6JFTHDWN4WFBSLPQXI2PQFVZHNTXMOG25UYZOCTIE6LMQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.917060331 (15h48m31.70040043s) [noise=true , address=212.102.52.214:4360 , piece_id=73PXBGVXO5KGNJDVS75DBD56COZQYSP26RBS5GND2BLG6JQLMXZA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.918744335 (15h48m31.702084434s) [noise=true , address=212.102.52.215:4386 , piece_id=SJIDT4PIAIMA7QFS64FI6FMNRRO6Q5S4PVQGUKPJQBPJ7N7ZFL2A, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.924271771 (15h48m31.70761187s) [noise=true , address=212.102.52.216:4408 , piece_id=EPE6MXFOZTO5PT3SLA3U6HI6UJWMNBPWAPI7LLGPB35CRKB3AZOA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.925897152 (15h48m31.709237251s) [noise=true , address=212.102.52.215:4382 , piece_id=JCRZDFY2O6CLCTURWS7KRMWQSDNOURAA3GLPKB4P2QO3UUNB26SA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.926060985 (15h48m31.709401084s) [noise=true , address=212.102.52.217:4435 , piece_id=RNBPLJS4P5H4B24FX7XIFKRUUAWTH3PFEA3JOYAS4LCNWUUKGSBQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.927768653 (15h48m31.711108752s) [noise=true , address=212.102.52.215:4376 , piece_id=25KT756B5NJCE44RWE22WJ7O5VXP7VJR6YO3M3LTRFX3CIYPVUQA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.929649962 (15h48m31.712990061s) [noise=true , address=212.102.52.214:4364 , piece_id=MU7IJZCB752M4M7LFCEJZCDEJJY6S5M63KH4YBSIIQN5LPHW6YQA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.957068207 (15h48m31.740408307s) [noise=true , address=212.102.52.215:4380 , piece_id=2S2FDBIAGAP7AMF3CL54BECVIGDEXDDG5OHT23FJKWPSVOUPN6RA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.957729358 (15h48m31.741069457s) [noise=true , address=212.102.52.213:4324 , piece_id=YONLVC2ARQPU3ZJOSJSG2WX6SOA65DJYA5GMIEQ5ZJ5DUPR6UEFQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959196257 (15h48m31.742536356s) [noise=true , address=212.102.52.217:4426 , piece_id=EXMYVMU7ZI4ZVELF2EMJNOZGDZAB4QZLJZ3M5DBNKIP433NFCUTA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959726810 (15h48m31.743066909s) [noise=true , address=212.102.52.217:4424 , piece_id=FLGYPCAZR5EF3FDQ5MT7CHBA5HDLM4CQOS6HCPVML2XKRLFGOGGQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959735406 (15h48m31.743075505s) [seg_pos={0 0}, upload=15135]: Satisfied cohort requirements after 16 pieces
May 25 17:52:45.959899980 (15h48m31.743240079s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MbNn3rECMrMhp1uJJRHy8v3DZkKaBEcZGwPMGumvPUvvKu5gM): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959906273 (15h48m31.743246372s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.959965084 (15h48m31.743305183s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eyfVMwFVb4Q5nbc54PSLebuyWfm2rCgCfGrNc6AovSWqe6Upn): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959969824 (15h48m31.743309923s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.959978169 (15h48m31.743318269s) [noise=true , address=212.102.52.214:4356 , piece_id=UAHRGRD6YLJZOMHHBNHHONX5UZBS5W5MJETFZJZMBBP3LPJ3CP5A, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1263MPtQZy9iB1pxhBYqZ9tgqitVmjm2X7gyehcdSRja9kWfPKd): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959986315 (15h48m31.743326414s) [noise=true , address=212.102.52.214:4356 , piece_id=UAHRGRD6YLJZOMHHBNHHONX5UZBS5W5MJETFZJZMBBP3LPJ3CP5A, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.960111764 (15h48m31.743451863s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Tyfw7Bp6ma6jTKyBJaSfr5u8w2CwExJVAwkeabjMnsHWWsDdf): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.960116693 (15h48m31.743456792s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.960130370 (15h48m31.743470469s) [seg_pos={0 0}, upload=15135]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 17:52:45.960200974 (15h48m31.743541073s) [upload=15135]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 17:52:45.960205563 (15h48m31.743545662s) [upload=15135]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 17:52:48.787245030 (15h48m34.570585129s) [upload=15135]: Done uploading segments...
May 25 17:52:48.787262443 (15h48m34.570602542s) [upload=15135]: Upload finished. err: <nil>
Duration: 4.100359123s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
677.441333ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7e710747942acdb2
FAIL (2026-05-25 18:01:03 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 677.459727ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
4.165667152s trace:
https://jaeger.dev.storj.io/trace/a5682f02df33c7c
FAIL (2026-05-25 18:00:42 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ZCYA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543483940 (15h48m30.326824029s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543527172 (15h48m30.326867271s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543870378 (15h48m30.327210477s) [noise=true , address=212.102.52.216:4408 , piece_id=EPE6MXFOZTO5PT3SLA3U6HI6UJWMNBPWAPI7LLGPB35CRKB3AZOA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.544411289 (15h48m30.327751388s) [noise=true , address=212.102.52.214:4364 , piece_id=MU7IJZCB752M4M7LFCEJZCDEJJY6S5M63KH4YBSIIQN5LPHW6YQA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.544595600 (15h48m30.327935699s) [noise=true , address=212.102.52.215:4376 , piece_id=25KT756B5NJCE44RWE22WJ7O5VXP7VJR6YO3M3LTRFX3CIYPVUQA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.545402781 (15h48m30.32874288s) [noise=true , address=212.102.52.215:4382 , piece_id=JCRZDFY2O6CLCTURWS7KRMWQSDNOURAA3GLPKB4P2QO3UUNB26SA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.545463316 (15h48m30.328803415s) [noise=true , address=212.102.52.214:4360 , piece_id=73PXBGVXO5KGNJDVS75DBD56COZQYSP26RBS5GND2BLG6JQLMXZA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546346461 (15h48m30.32968656s) [noise=true , address=212.102.52.216:4402 , piece_id=B5JT2RXBOTI4IUOCRZKPKA6LY6C3IGQGHWWWOXJ5FCYC64EOBIEQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546403570 (15h48m30.329743669s) [noise=true , address=212.102.52.217:4424 , piece_id=FLGYPCAZR5EF3FDQ5MT7CHBA5HDLM4CQOS6HCPVML2XKRLFGOGGQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546993276 (15h48m30.330333375s) [noise=true , address=212.102.52.215:4386 , piece_id=SJIDT4PIAIMA7QFS64FI6FMNRRO6Q5S4PVQGUKPJQBPJ7N7ZFL2A, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.548021087 (15h48m30.331361186s) [noise=true , address=212.102.52.216:4396 , piece_id=RYO7RQ26ZTRMDK22XCED2JM2K7WPEI57X25HMT5LQ73RIE7WTTYA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.548283338 (15h48m30.331623437s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.549042276 (15h48m30.332382365s) [noise=true , address=212.102.52.213:4336 , piece_id=MGI6NLZCQPDNW5I42M4NL5FWKWNOU7RFW4TZJG2LTRZATP2C742Q, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.549395759 (15h48m30.332735858s) [noise=true , address=212.102.52.217:4435 , piece_id=RNBPLJS4P5H4B24FX7XIFKRUUAWTH3PFEA3JOYAS4LCNWUUKGSBQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.550129380 (15h48m30.333469479s) [noise=true , address=212.102.52.217:4426 , piece_id=EXMYVMU7ZI4ZVELF2EMJNOZGDZAB4QZLJZ3M5DBNKIP433NFCUTA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:45.613527985 (15h48m31.396868084s) [upload=15135]: Next returned nil segment
May 25 17:52:45.613540610 (15h48m31.396880709s) [upload=15135]: Waiting for error group managing segments...
May 25 17:52:45.894196156 (15h48m31.677536255s) [noise=true , address=212.102.52.216:4394 , piece_id=4P3VUGG3TJGW6YENC6WC4J2EHG6TQNDMPXUHROT5BHRU3246ZCYA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.902564564 (15h48m31.685904663s) [noise=true , address=212.102.52.213:4336 , piece_id=MGI6NLZCQPDNW5I42M4NL5FWKWNOU7RFW4TZJG2LTRZATP2C742Q, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.902884974 (15h48m31.686225073s) [noise=true , address=212.102.52.216:4396 , piece_id=RYO7RQ26ZTRMDK22XCED2JM2K7WPEI57X25HMT5LQ73RIE7WTTYA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.903550163 (15h48m31.686890263s) [noise=true , address=212.102.52.216:4402 , piece_id=B5JT2RXBOTI4IUOCRZKPKA6LY6C3IGQGHWWWOXJ5FCYC64EOBIEQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.913910772 (15h48m31.697250881s) [noise=true , address=212.102.52.213:4333 , piece_id=TJKMUCX6JFTHDWN4WFBSLPQXI2PQFVZHNTXMOG25UYZOCTIE6LMQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.917060331 (15h48m31.70040043s) [noise=true , address=212.102.52.214:4360 , piece_id=73PXBGVXO5KGNJDVS75DBD56COZQYSP26RBS5GND2BLG6JQLMXZA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.918744335 (15h48m31.702084434s) [noise=true , address=212.102.52.215:4386 , piece_id=SJIDT4PIAIMA7QFS64FI6FMNRRO6Q5S4PVQGUKPJQBPJ7N7ZFL2A, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.924271771 (15h48m31.70761187s) [noise=true , address=212.102.52.216:4408 , piece_id=EPE6MXFOZTO5PT3SLA3U6HI6UJWMNBPWAPI7LLGPB35CRKB3AZOA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.925897152 (15h48m31.709237251s) [noise=true , address=212.102.52.215:4382 , piece_id=JCRZDFY2O6CLCTURWS7KRMWQSDNOURAA3GLPKB4P2QO3UUNB26SA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.926060985 (15h48m31.709401084s) [noise=true , address=212.102.52.217:4435 , piece_id=RNBPLJS4P5H4B24FX7XIFKRUUAWTH3PFEA3JOYAS4LCNWUUKGSBQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.927768653 (15h48m31.711108752s) [noise=true , address=212.102.52.215:4376 , piece_id=25KT756B5NJCE44RWE22WJ7O5VXP7VJR6YO3M3LTRFX3CIYPVUQA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.929649962 (15h48m31.712990061s) [noise=true , address=212.102.52.214:4364 , piece_id=MU7IJZCB752M4M7LFCEJZCDEJJY6S5M63KH4YBSIIQN5LPHW6YQA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.957068207 (15h48m31.740408307s) [noise=true , address=212.102.52.215:4380 , piece_id=2S2FDBIAGAP7AMF3CL54BECVIGDEXDDG5OHT23FJKWPSVOUPN6RA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.957729358 (15h48m31.741069457s) [noise=true , address=212.102.52.213:4324 , piece_id=YONLVC2ARQPU3ZJOSJSG2WX6SOA65DJYA5GMIEQ5ZJ5DUPR6UEFQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959196257 (15h48m31.742536356s) [noise=true , address=212.102.52.217:4426 , piece_id=EXMYVMU7ZI4ZVELF2EMJNOZGDZAB4QZLJZ3M5DBNKIP433NFCUTA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959726810 (15h48m31.743066909s) [noise=true , address=212.102.52.217:4424 , piece_id=FLGYPCAZR5EF3FDQ5MT7CHBA5HDLM4CQOS6HCPVML2XKRLFGOGGQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959735406 (15h48m31.743075505s) [seg_pos={0 0}, upload=15135]: Satisfied cohort requirements after 16 pieces
May 25 17:52:45.959899980 (15h48m31.743240079s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MbNn3rECMrMhp1uJJRHy8v3DZkKaBEcZGwPMGumvPUvvKu5gM): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959906273 (15h48m31.743246372s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.959965084 (15h48m31.743305183s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eyfVMwFVb4Q5nbc54PSLebuyWfm2rCgCfGrNc6AovSWqe6Upn): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959969824 (15h48m31.743309923s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.959978169 (15h48m31.743318269s) [noise=true , address=212.102.52.214:4356 , piece_id=UAHRGRD6YLJZOMHHBNHHONX5UZBS5W5MJETFZJZMBBP3LPJ3CP5A, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1263MPtQZy9iB1pxhBYqZ9tgqitVmjm2X7gyehcdSRja9kWfPKd): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959986315 (15h48m31.743326414s) [noise=true , address=212.102.52.214:4356 , piece_id=UAHRGRD6YLJZOMHHBNHHONX5UZBS5W5MJETFZJZMBBP3LPJ3CP5A, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.960111764 (15h48m31.743451863s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Tyfw7Bp6ma6jTKyBJaSfr5u8w2CwExJVAwkeabjMnsHWWsDdf): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.960116693 (15h48m31.743456792s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.960130370 (15h48m31.743470469s) [seg_pos={0 0}, upload=15135]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 17:52:45.960200974 (15h48m31.743541073s) [upload=15135]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 17:52:45.960205563 (15h48m31.743545662s) [upload=15135]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 17:52:48.787245030 (15h48m34.570585129s) [upload=15135]: Done uploading segments...
May 25 17:52:48.787262443 (15h48m34.570602542s) [upload=15135]: Upload finished. err: <nil>
Duration: 4.165691197s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.............X..........................................X...] duration:
3.870789795s trace:
https://jaeger.dev.storj.io/trace/14eb8131f680ed75
FAIL (2026-05-25 18:00:35 UTC)
orbiter warning: context deadline exceeded
Duration: 3.870809433s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
10 out of 60 is failed (threshold=3) [.......X......X................X....X.......X......X..XX.XX.] duration:
55.09785262s trace:
https://jaeger.dev.storj.io/trace/3fbd8e4b1d1c6db8
FAIL (2026-05-25 18:00:18 UTC)
orbiter warning: context deadline exceeded
Duration: 55.097960114s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
672.764086ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7e710747942acdb2
FAIL (2026-05-25 17:59:59 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 672.787381ms
core
satellite
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [X.............X..........................................X..] duration:
4.500651037s trace:
https://jaeger.dev.storj.io/trace/14eb8131f680ed75
FAIL (2026-05-25 17:59:58 UTC)
orbiter warning: context deadline exceeded
Duration: 4.500673471s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
6.009665358s trace:
https://jaeger.dev.storj.io/trace/a5682f02df33c7c
FAIL (2026-05-25 17:59:41 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ZCYA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543483940 (15h48m30.326824029s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543527172 (15h48m30.326867271s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543870378 (15h48m30.327210477s) [noise=true , address=212.102.52.216:4408 , piece_id=EPE6MXFOZTO5PT3SLA3U6HI6UJWMNBPWAPI7LLGPB35CRKB3AZOA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.544411289 (15h48m30.327751388s) [noise=true , address=212.102.52.214:4364 , piece_id=MU7IJZCB752M4M7LFCEJZCDEJJY6S5M63KH4YBSIIQN5LPHW6YQA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.544595600 (15h48m30.327935699s) [noise=true , address=212.102.52.215:4376 , piece_id=25KT756B5NJCE44RWE22WJ7O5VXP7VJR6YO3M3LTRFX3CIYPVUQA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.545402781 (15h48m30.32874288s) [noise=true , address=212.102.52.215:4382 , piece_id=JCRZDFY2O6CLCTURWS7KRMWQSDNOURAA3GLPKB4P2QO3UUNB26SA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.545463316 (15h48m30.328803415s) [noise=true , address=212.102.52.214:4360 , piece_id=73PXBGVXO5KGNJDVS75DBD56COZQYSP26RBS5GND2BLG6JQLMXZA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546346461 (15h48m30.32968656s) [noise=true , address=212.102.52.216:4402 , piece_id=B5JT2RXBOTI4IUOCRZKPKA6LY6C3IGQGHWWWOXJ5FCYC64EOBIEQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546403570 (15h48m30.329743669s) [noise=true , address=212.102.52.217:4424 , piece_id=FLGYPCAZR5EF3FDQ5MT7CHBA5HDLM4CQOS6HCPVML2XKRLFGOGGQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546993276 (15h48m30.330333375s) [noise=true , address=212.102.52.215:4386 , piece_id=SJIDT4PIAIMA7QFS64FI6FMNRRO6Q5S4PVQGUKPJQBPJ7N7ZFL2A, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.548021087 (15h48m30.331361186s) [noise=true , address=212.102.52.216:4396 , piece_id=RYO7RQ26ZTRMDK22XCED2JM2K7WPEI57X25HMT5LQ73RIE7WTTYA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.548283338 (15h48m30.331623437s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.549042276 (15h48m30.332382365s) [noise=true , address=212.102.52.213:4336 , piece_id=MGI6NLZCQPDNW5I42M4NL5FWKWNOU7RFW4TZJG2LTRZATP2C742Q, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.549395759 (15h48m30.332735858s) [noise=true , address=212.102.52.217:4435 , piece_id=RNBPLJS4P5H4B24FX7XIFKRUUAWTH3PFEA3JOYAS4LCNWUUKGSBQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.550129380 (15h48m30.333469479s) [noise=true , address=212.102.52.217:4426 , piece_id=EXMYVMU7ZI4ZVELF2EMJNOZGDZAB4QZLJZ3M5DBNKIP433NFCUTA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:45.613527985 (15h48m31.396868084s) [upload=15135]: Next returned nil segment
May 25 17:52:45.613540610 (15h48m31.396880709s) [upload=15135]: Waiting for error group managing segments...
May 25 17:52:45.894196156 (15h48m31.677536255s) [noise=true , address=212.102.52.216:4394 , piece_id=4P3VUGG3TJGW6YENC6WC4J2EHG6TQNDMPXUHROT5BHRU3246ZCYA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.902564564 (15h48m31.685904663s) [noise=true , address=212.102.52.213:4336 , piece_id=MGI6NLZCQPDNW5I42M4NL5FWKWNOU7RFW4TZJG2LTRZATP2C742Q, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.902884974 (15h48m31.686225073s) [noise=true , address=212.102.52.216:4396 , piece_id=RYO7RQ26ZTRMDK22XCED2JM2K7WPEI57X25HMT5LQ73RIE7WTTYA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.903550163 (15h48m31.686890263s) [noise=true , address=212.102.52.216:4402 , piece_id=B5JT2RXBOTI4IUOCRZKPKA6LY6C3IGQGHWWWOXJ5FCYC64EOBIEQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.913910772 (15h48m31.697250881s) [noise=true , address=212.102.52.213:4333 , piece_id=TJKMUCX6JFTHDWN4WFBSLPQXI2PQFVZHNTXMOG25UYZOCTIE6LMQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.917060331 (15h48m31.70040043s) [noise=true , address=212.102.52.214:4360 , piece_id=73PXBGVXO5KGNJDVS75DBD56COZQYSP26RBS5GND2BLG6JQLMXZA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.918744335 (15h48m31.702084434s) [noise=true , address=212.102.52.215:4386 , piece_id=SJIDT4PIAIMA7QFS64FI6FMNRRO6Q5S4PVQGUKPJQBPJ7N7ZFL2A, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.924271771 (15h48m31.70761187s) [noise=true , address=212.102.52.216:4408 , piece_id=EPE6MXFOZTO5PT3SLA3U6HI6UJWMNBPWAPI7LLGPB35CRKB3AZOA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.925897152 (15h48m31.709237251s) [noise=true , address=212.102.52.215:4382 , piece_id=JCRZDFY2O6CLCTURWS7KRMWQSDNOURAA3GLPKB4P2QO3UUNB26SA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.926060985 (15h48m31.709401084s) [noise=true , address=212.102.52.217:4435 , piece_id=RNBPLJS4P5H4B24FX7XIFKRUUAWTH3PFEA3JOYAS4LCNWUUKGSBQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.927768653 (15h48m31.711108752s) [noise=true , address=212.102.52.215:4376 , piece_id=25KT756B5NJCE44RWE22WJ7O5VXP7VJR6YO3M3LTRFX3CIYPVUQA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.929649962 (15h48m31.712990061s) [noise=true , address=212.102.52.214:4364 , piece_id=MU7IJZCB752M4M7LFCEJZCDEJJY6S5M63KH4YBSIIQN5LPHW6YQA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.957068207 (15h48m31.740408307s) [noise=true , address=212.102.52.215:4380 , piece_id=2S2FDBIAGAP7AMF3CL54BECVIGDEXDDG5OHT23FJKWPSVOUPN6RA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.957729358 (15h48m31.741069457s) [noise=true , address=212.102.52.213:4324 , piece_id=YONLVC2ARQPU3ZJOSJSG2WX6SOA65DJYA5GMIEQ5ZJ5DUPR6UEFQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959196257 (15h48m31.742536356s) [noise=true , address=212.102.52.217:4426 , piece_id=EXMYVMU7ZI4ZVELF2EMJNOZGDZAB4QZLJZ3M5DBNKIP433NFCUTA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959726810 (15h48m31.743066909s) [noise=true , address=212.102.52.217:4424 , piece_id=FLGYPCAZR5EF3FDQ5MT7CHBA5HDLM4CQOS6HCPVML2XKRLFGOGGQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959735406 (15h48m31.743075505s) [seg_pos={0 0}, upload=15135]: Satisfied cohort requirements after 16 pieces
May 25 17:52:45.959899980 (15h48m31.743240079s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MbNn3rECMrMhp1uJJRHy8v3DZkKaBEcZGwPMGumvPUvvKu5gM): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959906273 (15h48m31.743246372s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.959965084 (15h48m31.743305183s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eyfVMwFVb4Q5nbc54PSLebuyWfm2rCgCfGrNc6AovSWqe6Upn): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959969824 (15h48m31.743309923s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.959978169 (15h48m31.743318269s) [noise=true , address=212.102.52.214:4356 , piece_id=UAHRGRD6YLJZOMHHBNHHONX5UZBS5W5MJETFZJZMBBP3LPJ3CP5A, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1263MPtQZy9iB1pxhBYqZ9tgqitVmjm2X7gyehcdSRja9kWfPKd): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959986315 (15h48m31.743326414s) [noise=true , address=212.102.52.214:4356 , piece_id=UAHRGRD6YLJZOMHHBNHHONX5UZBS5W5MJETFZJZMBBP3LPJ3CP5A, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.960111764 (15h48m31.743451863s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Tyfw7Bp6ma6jTKyBJaSfr5u8w2CwExJVAwkeabjMnsHWWsDdf): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.960116693 (15h48m31.743456792s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.960130370 (15h48m31.743470469s) [seg_pos={0 0}, upload=15135]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 17:52:45.960200974 (15h48m31.743541073s) [upload=15135]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 17:52:45.960205563 (15h48m31.743545662s) [upload=15135]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 17:52:48.787245030 (15h48m34.570585129s) [upload=15135]: Done uploading segments...
May 25 17:52:48.787262443 (15h48m34.570602542s) [upload=15135]: Upload finished. err: <nil>
Duration: 6.009681838s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
10 out of 60 is failed (threshold=3) [........X......X................X....X.......X......X..XX.XX] duration:
1m0.000866617s trace:
https://jaeger.dev.storj.io/trace/3fbd8e4b1d1c6db8
FAIL (2026-05-25 17:59:23 UTC)
orbiter warning: context deadline exceeded
Duration: 1m0.000874282s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
675.510938ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7e710747942acdb2
FAIL (2026-05-25 17:59:00 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 675.536777ms
core
satellite
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [.X.............X..........................................X.] duration:
4.670072337s trace:
https://jaeger.dev.storj.io/trace/14eb8131f680ed75
FAIL (2026-05-25 17:58:49 UTC)
orbiter warning: context deadline exceeded
Duration: 4.670096844s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
4.059232214s trace:
https://jaeger.dev.storj.io/trace/a5682f02df33c7c
FAIL (2026-05-25 17:58:30 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
ZCYA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543483940 (15h48m30.326824029s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543527172 (15h48m30.326867271s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.543870378 (15h48m30.327210477s) [noise=true , address=212.102.52.216:4408 , piece_id=EPE6MXFOZTO5PT3SLA3U6HI6UJWMNBPWAPI7LLGPB35CRKB3AZOA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.544411289 (15h48m30.327751388s) [noise=true , address=212.102.52.214:4364 , piece_id=MU7IJZCB752M4M7LFCEJZCDEJJY6S5M63KH4YBSIIQN5LPHW6YQA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.544595600 (15h48m30.327935699s) [noise=true , address=212.102.52.215:4376 , piece_id=25KT756B5NJCE44RWE22WJ7O5VXP7VJR6YO3M3LTRFX3CIYPVUQA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.545402781 (15h48m30.32874288s) [noise=true , address=212.102.52.215:4382 , piece_id=JCRZDFY2O6CLCTURWS7KRMWQSDNOURAA3GLPKB4P2QO3UUNB26SA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.545463316 (15h48m30.328803415s) [noise=true , address=212.102.52.214:4360 , piece_id=73PXBGVXO5KGNJDVS75DBD56COZQYSP26RBS5GND2BLG6JQLMXZA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546346461 (15h48m30.32968656s) [noise=true , address=212.102.52.216:4402 , piece_id=B5JT2RXBOTI4IUOCRZKPKA6LY6C3IGQGHWWWOXJ5FCYC64EOBIEQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546403570 (15h48m30.329743669s) [noise=true , address=212.102.52.217:4424 , piece_id=FLGYPCAZR5EF3FDQ5MT7CHBA5HDLM4CQOS6HCPVML2XKRLFGOGGQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.546993276 (15h48m30.330333375s) [noise=true , address=212.102.52.215:4386 , piece_id=SJIDT4PIAIMA7QFS64FI6FMNRRO6Q5S4PVQGUKPJQBPJ7N7ZFL2A, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.548021087 (15h48m30.331361186s) [noise=true , address=212.102.52.216:4396 , piece_id=RYO7RQ26ZTRMDK22XCED2JM2K7WPEI57X25HMT5LQ73RIE7WTTYA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.548283338 (15h48m30.331623437s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.549042276 (15h48m30.332382365s) [noise=true , address=212.102.52.213:4336 , piece_id=MGI6NLZCQPDNW5I42M4NL5FWKWNOU7RFW4TZJG2LTRZATP2C742Q, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.549395759 (15h48m30.332735858s) [noise=true , address=212.102.52.217:4435 , piece_id=RNBPLJS4P5H4B24FX7XIFKRUUAWTH3PFEA3JOYAS4LCNWUUKGSBQ, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:44.550129380 (15h48m30.333469479s) [noise=true , address=212.102.52.217:4426 , piece_id=EXMYVMU7ZI4ZVELF2EMJNOZGDZAB4QZLJZ3M5DBNKIP433NFCUTA, seg_pos={0 0}, upload=15135]: Uploading piece...
May 25 17:52:45.613527985 (15h48m31.396868084s) [upload=15135]: Next returned nil segment
May 25 17:52:45.613540610 (15h48m31.396880709s) [upload=15135]: Waiting for error group managing segments...
May 25 17:52:45.894196156 (15h48m31.677536255s) [noise=true , address=212.102.52.216:4394 , piece_id=4P3VUGG3TJGW6YENC6WC4J2EHG6TQNDMPXUHROT5BHRU3246ZCYA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.902564564 (15h48m31.685904663s) [noise=true , address=212.102.52.213:4336 , piece_id=MGI6NLZCQPDNW5I42M4NL5FWKWNOU7RFW4TZJG2LTRZATP2C742Q, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.902884974 (15h48m31.686225073s) [noise=true , address=212.102.52.216:4396 , piece_id=RYO7RQ26ZTRMDK22XCED2JM2K7WPEI57X25HMT5LQ73RIE7WTTYA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.903550163 (15h48m31.686890263s) [noise=true , address=212.102.52.216:4402 , piece_id=B5JT2RXBOTI4IUOCRZKPKA6LY6C3IGQGHWWWOXJ5FCYC64EOBIEQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.913910772 (15h48m31.697250881s) [noise=true , address=212.102.52.213:4333 , piece_id=TJKMUCX6JFTHDWN4WFBSLPQXI2PQFVZHNTXMOG25UYZOCTIE6LMQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.917060331 (15h48m31.70040043s) [noise=true , address=212.102.52.214:4360 , piece_id=73PXBGVXO5KGNJDVS75DBD56COZQYSP26RBS5GND2BLG6JQLMXZA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.918744335 (15h48m31.702084434s) [noise=true , address=212.102.52.215:4386 , piece_id=SJIDT4PIAIMA7QFS64FI6FMNRRO6Q5S4PVQGUKPJQBPJ7N7ZFL2A, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.924271771 (15h48m31.70761187s) [noise=true , address=212.102.52.216:4408 , piece_id=EPE6MXFOZTO5PT3SLA3U6HI6UJWMNBPWAPI7LLGPB35CRKB3AZOA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.925897152 (15h48m31.709237251s) [noise=true , address=212.102.52.215:4382 , piece_id=JCRZDFY2O6CLCTURWS7KRMWQSDNOURAA3GLPKB4P2QO3UUNB26SA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.926060985 (15h48m31.709401084s) [noise=true , address=212.102.52.217:4435 , piece_id=RNBPLJS4P5H4B24FX7XIFKRUUAWTH3PFEA3JOYAS4LCNWUUKGSBQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.927768653 (15h48m31.711108752s) [noise=true , address=212.102.52.215:4376 , piece_id=25KT756B5NJCE44RWE22WJ7O5VXP7VJR6YO3M3LTRFX3CIYPVUQA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.929649962 (15h48m31.712990061s) [noise=true , address=212.102.52.214:4364 , piece_id=MU7IJZCB752M4M7LFCEJZCDEJJY6S5M63KH4YBSIIQN5LPHW6YQA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.957068207 (15h48m31.740408307s) [noise=true , address=212.102.52.215:4380 , piece_id=2S2FDBIAGAP7AMF3CL54BECVIGDEXDDG5OHT23FJKWPSVOUPN6RA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.957729358 (15h48m31.741069457s) [noise=true , address=212.102.52.213:4324 , piece_id=YONLVC2ARQPU3ZJOSJSG2WX6SOA65DJYA5GMIEQ5ZJ5DUPR6UEFQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959196257 (15h48m31.742536356s) [noise=true , address=212.102.52.217:4426 , piece_id=EXMYVMU7ZI4ZVELF2EMJNOZGDZAB4QZLJZ3M5DBNKIP433NFCUTA, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959726810 (15h48m31.743066909s) [noise=true , address=212.102.52.217:4424 , piece_id=FLGYPCAZR5EF3FDQ5MT7CHBA5HDLM4CQOS6HCPVML2XKRLFGOGGQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: <nil>
May 25 17:52:45.959735406 (15h48m31.743075505s) [seg_pos={0 0}, upload=15135]: Satisfied cohort requirements after 16 pieces
May 25 17:52:45.959899980 (15h48m31.743240079s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MbNn3rECMrMhp1uJJRHy8v3DZkKaBEcZGwPMGumvPUvvKu5gM): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959906273 (15h48m31.743246372s) [noise=true , address=212.102.52.214:4366 , piece_id=G45337QJCPJ6TJ7QSOIDOQ3XSVKBORJRBJ5R2PBZO4VP6PIQTYDQ, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.959965084 (15h48m31.743305183s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eyfVMwFVb4Q5nbc54PSLebuyWfm2rCgCfGrNc6AovSWqe6Upn): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959969824 (15h48m31.743309923s) [noise=true , address=212.102.52.213:4331 , piece_id=4RH2RC7G3WEFM6PU55HEMTR5YUE3XDCUW7FRS33CFTP444NRYVPQ, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.959978169 (15h48m31.743318269s) [noise=true , address=212.102.52.214:4356 , piece_id=UAHRGRD6YLJZOMHHBNHHONX5UZBS5W5MJETFZJZMBBP3LPJ3CP5A, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1263MPtQZy9iB1pxhBYqZ9tgqitVmjm2X7gyehcdSRja9kWfPKd): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.959986315 (15h48m31.743326414s) [noise=true , address=212.102.52.214:4356 , piece_id=UAHRGRD6YLJZOMHHBNHHONX5UZBS5W5MJETFZJZMBBP3LPJ3CP5A, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.960111764 (15h48m31.743451863s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Tyfw7Bp6ma6jTKyBJaSfr5u8w2CwExJVAwkeabjMnsHWWsDdf): protocol: expected piece hash; context canceled; EOF
May 25 17:52:45.960116693 (15h48m31.743456792s) [noise=true , address=212.102.52.217:4429 , piece_id=N3ZO2XCFAONIWEXH3OW36LOMSJ5GYTTHDZUY6VMG3PCCO6TEUS6Q, seg_pos={0 0}, upload=15135]: Long tail cancellation due to optimal threshold
May 25 17:52:45.960130370 (15h48m31.743470469s) [seg_pos={0 0}, upload=15135]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 17:52:45.960200974 (15h48m31.743541073s) [upload=15135]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 17:52:45.960205563 (15h48m31.743545662s) [upload=15135]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 17:52:48.787245030 (15h48m34.570585129s) [upload=15135]: Done uploading segments...
May 25 17:52:48.787262443 (15h48m34.570602542s) [upload=15135]: Upload finished. err: <nil>
Duration: 4.059255228s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [..X.............X..........................................X] duration:
5.001616553s trace:
https://jaeger.dev.storj.io/trace/14eb8131f680ed75
FAIL (2026-05-25 17:58:23 UTC)
orbiter warning: context deadline exceeded
Duration: 5.001634447s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
9 out of 60 is failed (threshold=3) [.........X......X................X....X.......X......X..XX.X] duration:
1m0.000870543s trace:
https://jaeger.dev.storj.io/trace/6acdf63f1d0d8d3d
FAIL (2026-05-25 17:58:23 UTC)
orbiter warning: context deadline exceeded
Duration: 1m0.00090093s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
677.054254ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7e710747942acdb2
FAIL (2026-05-25 17:58:22 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 677.080654ms
core
satellite
us1
|