Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
13 out of 60 is failed (threshold=3) [X....X.........X.XX....X..X.X....................X.X..XX.X..] duration:
25.967933284s trace:
https://jaeger.dev.storj.io/trace/20510f10ce4ee74b
FAIL (2026-05-25 23:49:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 25.967952631s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
5.514513169s trace:
https://jaeger.dev.storj.io/trace/154f04129c3ba223
FAIL (2026-05-25 23:48:12 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
H5UA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.669978445 (21h21m58.453318544s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670570324 (21h21m58.453910423s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670977932 (21h21m58.454318031s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671082672 (21h21m58.454422771s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671399046 (21h21m58.454739145s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671505018 (21h21m58.454845117s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671768601 (21h21m58.4551087s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671873380 (21h21m58.455213479s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672172411 (21h21m58.45551251s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672225192 (21h21m58.455565291s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672604115 (21h21m58.455944214s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672647878 (21h21m58.455987977s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672979411 (21h21m58.45631951s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673026311 (21h21m58.45636641s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673319009 (21h21m58.456659108s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:13.747141337 (21h21m59.530481436s) [upload=20458]: Next returned nil segment
May 25 23:26:13.747155513 (21h21m59.530495612s) [upload=20458]: Waiting for error group managing segments...
May 25 23:26:14.031706029 (21h21m59.815046128s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034098815 (21h21m59.817438914s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034662339 (21h21m59.818002439s) [noise=true , address=212.102.52.214:4362 , piece_id=ZSAFYLTPN3ESF7KDM6RXVTPIHI2O5BJ77WJQXDRGXEIM5IUTEVMQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034708708 (21h21m59.818048807s) [noise=true , address=212.102.52.217:4419 , piece_id=3IAS5VH7QJ2SJOMUCY2B4MCNYECWLHUH3AGH6MOGM4NS6KDAH5UA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035071280 (21h21m59.818411379s) [noise=true , address=212.102.52.217:4424 , piece_id=OHAYPIY666MWXWJTMWLPWTPE3RAXI6SG6NRFZRQS7VO4GKYCOECQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035492163 (21h21m59.818832272s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.036495778 (21h21m59.819835877s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.037130078 (21h21m59.820470177s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.040472716 (21h21m59.823812815s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.042158933 (21h21m59.825499032s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045074927 (21h21m59.828415026s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045309063 (21h21m59.828649162s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046163785 (21h21m59.829503874s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046705198 (21h21m59.830045297s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.062034446 (21h21m59.845374546s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070397062 (21h21m59.853737161s) [noise=true , address=212.102.52.213:4322 , piece_id=TW24WKT2HYX65MOG6ILC2GYJPHYS5VRAEKPWGNF3MMPVJYMAZPHQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070415798 (21h21m59.853755897s) [seg_pos={0 0}, upload=20458]: Satisfied cohort requirements after 16 pieces
May 25 23:26:14.070581143 (21h21m59.853921242s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070588727 (21h21m59.853928826s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070817384 (21h21m59.854157483s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cCanaWd2hWD8vcQCophdGPopVihkjKSuUcB7ZyqqBcK1JTvSf): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070822854 (21h21m59.854162943s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070908688 (21h21m59.854248787s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YyuYGM7ttMRXybutoxVQBktQPes1XMce5fTs4BUfQVykeZtqT): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070913157 (21h21m59.854253256s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071162232 (21h21m59.854502341s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BL6sUUowTDCFa3Zu7Xv5SheJRFGzEHVd1yq3goxmvT5WggLg4): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.071169757 (21h21m59.854509866s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071187790 (21h21m59.854527889s) [seg_pos={0 0}, upload=20458]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 23:26:14.071292471 (21h21m59.85463257s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 23:26:14.071297069 (21h21m59.854637168s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 23:26:38.384275645 (21h22m24.167615734s) [upload=20458]: Done uploading segments...
May 25 23:26:38.384290073 (21h22m24.167630172s) [upload=20458]: Upload finished. err: <nil>
Duration: 5.514537434s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
13 out of 60 is failed (threshold=3) [.X....X.........X.XX....X..X.X....................X.X..XX.X.] duration:
6.542032476s trace:
https://jaeger.dev.storj.io/trace/20510f10ce4ee74b
FAIL (2026-05-25 23:48:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.542041753s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
13 out of 60 is failed (threshold=3) [..X....X.........X.XX....X..X.X....................X.X..XX.X] duration:
1m0.000850827s trace:
https://jaeger.dev.storj.io/trace/20510f10ce4ee74b
FAIL (2026-05-25 23:48:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000858923s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
3.958529268s trace:
https://jaeger.dev.storj.io/trace/154f04129c3ba223
FAIL (2026-05-25 23:47:19 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
H5UA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.669978445 (21h21m58.453318544s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670570324 (21h21m58.453910423s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670977932 (21h21m58.454318031s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671082672 (21h21m58.454422771s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671399046 (21h21m58.454739145s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671505018 (21h21m58.454845117s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671768601 (21h21m58.4551087s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671873380 (21h21m58.455213479s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672172411 (21h21m58.45551251s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672225192 (21h21m58.455565291s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672604115 (21h21m58.455944214s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672647878 (21h21m58.455987977s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672979411 (21h21m58.45631951s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673026311 (21h21m58.45636641s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673319009 (21h21m58.456659108s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:13.747141337 (21h21m59.530481436s) [upload=20458]: Next returned nil segment
May 25 23:26:13.747155513 (21h21m59.530495612s) [upload=20458]: Waiting for error group managing segments...
May 25 23:26:14.031706029 (21h21m59.815046128s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034098815 (21h21m59.817438914s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034662339 (21h21m59.818002439s) [noise=true , address=212.102.52.214:4362 , piece_id=ZSAFYLTPN3ESF7KDM6RXVTPIHI2O5BJ77WJQXDRGXEIM5IUTEVMQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034708708 (21h21m59.818048807s) [noise=true , address=212.102.52.217:4419 , piece_id=3IAS5VH7QJ2SJOMUCY2B4MCNYECWLHUH3AGH6MOGM4NS6KDAH5UA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035071280 (21h21m59.818411379s) [noise=true , address=212.102.52.217:4424 , piece_id=OHAYPIY666MWXWJTMWLPWTPE3RAXI6SG6NRFZRQS7VO4GKYCOECQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035492163 (21h21m59.818832272s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.036495778 (21h21m59.819835877s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.037130078 (21h21m59.820470177s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.040472716 (21h21m59.823812815s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.042158933 (21h21m59.825499032s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045074927 (21h21m59.828415026s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045309063 (21h21m59.828649162s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046163785 (21h21m59.829503874s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046705198 (21h21m59.830045297s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.062034446 (21h21m59.845374546s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070397062 (21h21m59.853737161s) [noise=true , address=212.102.52.213:4322 , piece_id=TW24WKT2HYX65MOG6ILC2GYJPHYS5VRAEKPWGNF3MMPVJYMAZPHQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070415798 (21h21m59.853755897s) [seg_pos={0 0}, upload=20458]: Satisfied cohort requirements after 16 pieces
May 25 23:26:14.070581143 (21h21m59.853921242s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070588727 (21h21m59.853928826s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070817384 (21h21m59.854157483s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cCanaWd2hWD8vcQCophdGPopVihkjKSuUcB7ZyqqBcK1JTvSf): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070822854 (21h21m59.854162943s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070908688 (21h21m59.854248787s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YyuYGM7ttMRXybutoxVQBktQPes1XMce5fTs4BUfQVykeZtqT): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070913157 (21h21m59.854253256s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071162232 (21h21m59.854502341s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BL6sUUowTDCFa3Zu7Xv5SheJRFGzEHVd1yq3goxmvT5WggLg4): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.071169757 (21h21m59.854509866s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071187790 (21h21m59.854527889s) [seg_pos={0 0}, upload=20458]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 23:26:14.071292471 (21h21m59.85463257s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 23:26:14.071297069 (21h21m59.854637168s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 23:26:38.384275645 (21h22m24.167615734s) [upload=20458]: Done uploading segments...
May 25 23:26:38.384290073 (21h22m24.167630172s) [upload=20458]: Upload finished. err: <nil>
Duration: 3.958550269s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
13 out of 60 is failed (threshold=3) [X..X....X.........X.XX....X..X.X....................X.X..XX.] duration:
58.88693008s trace:
https://jaeger.dev.storj.io/trace/69c3385d7a74cc97
FAIL (2026-05-25 23:47:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 58.886939557s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
6.573996025s trace:
https://jaeger.dev.storj.io/trace/154f04129c3ba223
FAIL (2026-05-25 23:46:53 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
H5UA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.669978445 (21h21m58.453318544s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670570324 (21h21m58.453910423s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670977932 (21h21m58.454318031s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671082672 (21h21m58.454422771s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671399046 (21h21m58.454739145s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671505018 (21h21m58.454845117s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671768601 (21h21m58.4551087s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671873380 (21h21m58.455213479s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672172411 (21h21m58.45551251s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672225192 (21h21m58.455565291s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672604115 (21h21m58.455944214s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672647878 (21h21m58.455987977s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672979411 (21h21m58.45631951s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673026311 (21h21m58.45636641s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673319009 (21h21m58.456659108s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:13.747141337 (21h21m59.530481436s) [upload=20458]: Next returned nil segment
May 25 23:26:13.747155513 (21h21m59.530495612s) [upload=20458]: Waiting for error group managing segments...
May 25 23:26:14.031706029 (21h21m59.815046128s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034098815 (21h21m59.817438914s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034662339 (21h21m59.818002439s) [noise=true , address=212.102.52.214:4362 , piece_id=ZSAFYLTPN3ESF7KDM6RXVTPIHI2O5BJ77WJQXDRGXEIM5IUTEVMQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034708708 (21h21m59.818048807s) [noise=true , address=212.102.52.217:4419 , piece_id=3IAS5VH7QJ2SJOMUCY2B4MCNYECWLHUH3AGH6MOGM4NS6KDAH5UA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035071280 (21h21m59.818411379s) [noise=true , address=212.102.52.217:4424 , piece_id=OHAYPIY666MWXWJTMWLPWTPE3RAXI6SG6NRFZRQS7VO4GKYCOECQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035492163 (21h21m59.818832272s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.036495778 (21h21m59.819835877s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.037130078 (21h21m59.820470177s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.040472716 (21h21m59.823812815s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.042158933 (21h21m59.825499032s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045074927 (21h21m59.828415026s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045309063 (21h21m59.828649162s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046163785 (21h21m59.829503874s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046705198 (21h21m59.830045297s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.062034446 (21h21m59.845374546s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070397062 (21h21m59.853737161s) [noise=true , address=212.102.52.213:4322 , piece_id=TW24WKT2HYX65MOG6ILC2GYJPHYS5VRAEKPWGNF3MMPVJYMAZPHQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070415798 (21h21m59.853755897s) [seg_pos={0 0}, upload=20458]: Satisfied cohort requirements after 16 pieces
May 25 23:26:14.070581143 (21h21m59.853921242s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070588727 (21h21m59.853928826s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070817384 (21h21m59.854157483s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cCanaWd2hWD8vcQCophdGPopVihkjKSuUcB7ZyqqBcK1JTvSf): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070822854 (21h21m59.854162943s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070908688 (21h21m59.854248787s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YyuYGM7ttMRXybutoxVQBktQPes1XMce5fTs4BUfQVykeZtqT): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070913157 (21h21m59.854253256s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071162232 (21h21m59.854502341s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BL6sUUowTDCFa3Zu7Xv5SheJRFGzEHVd1yq3goxmvT5WggLg4): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.071169757 (21h21m59.854509866s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071187790 (21h21m59.854527889s) [seg_pos={0 0}, upload=20458]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 23:26:14.071292471 (21h21m59.85463257s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 23:26:14.071297069 (21h21m59.854637168s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 23:26:38.384275645 (21h22m24.167615734s) [upload=20458]: Done uploading segments...
May 25 23:26:38.384290073 (21h22m24.167630172s) [upload=20458]: Upload finished. err: <nil>
Duration: 6.574017536s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
13 out of 60 is failed (threshold=3) [.X..X....X.........X.XX....X..X.X....................X.X..XX] duration:
1m0.000389831s trace:
https://jaeger.dev.storj.io/trace/69c3385d7a74cc97
FAIL (2026-05-25 23:46:05 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000398137s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
4.109179492s trace:
https://jaeger.dev.storj.io/trace/154f04129c3ba223
FAIL (2026-05-25 23:45:45 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
H5UA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.669978445 (21h21m58.453318544s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670570324 (21h21m58.453910423s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670977932 (21h21m58.454318031s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671082672 (21h21m58.454422771s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671399046 (21h21m58.454739145s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671505018 (21h21m58.454845117s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671768601 (21h21m58.4551087s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671873380 (21h21m58.455213479s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672172411 (21h21m58.45551251s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672225192 (21h21m58.455565291s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672604115 (21h21m58.455944214s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672647878 (21h21m58.455987977s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672979411 (21h21m58.45631951s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673026311 (21h21m58.45636641s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673319009 (21h21m58.456659108s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:13.747141337 (21h21m59.530481436s) [upload=20458]: Next returned nil segment
May 25 23:26:13.747155513 (21h21m59.530495612s) [upload=20458]: Waiting for error group managing segments...
May 25 23:26:14.031706029 (21h21m59.815046128s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034098815 (21h21m59.817438914s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034662339 (21h21m59.818002439s) [noise=true , address=212.102.52.214:4362 , piece_id=ZSAFYLTPN3ESF7KDM6RXVTPIHI2O5BJ77WJQXDRGXEIM5IUTEVMQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034708708 (21h21m59.818048807s) [noise=true , address=212.102.52.217:4419 , piece_id=3IAS5VH7QJ2SJOMUCY2B4MCNYECWLHUH3AGH6MOGM4NS6KDAH5UA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035071280 (21h21m59.818411379s) [noise=true , address=212.102.52.217:4424 , piece_id=OHAYPIY666MWXWJTMWLPWTPE3RAXI6SG6NRFZRQS7VO4GKYCOECQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035492163 (21h21m59.818832272s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.036495778 (21h21m59.819835877s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.037130078 (21h21m59.820470177s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.040472716 (21h21m59.823812815s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.042158933 (21h21m59.825499032s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045074927 (21h21m59.828415026s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045309063 (21h21m59.828649162s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046163785 (21h21m59.829503874s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046705198 (21h21m59.830045297s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.062034446 (21h21m59.845374546s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070397062 (21h21m59.853737161s) [noise=true , address=212.102.52.213:4322 , piece_id=TW24WKT2HYX65MOG6ILC2GYJPHYS5VRAEKPWGNF3MMPVJYMAZPHQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070415798 (21h21m59.853755897s) [seg_pos={0 0}, upload=20458]: Satisfied cohort requirements after 16 pieces
May 25 23:26:14.070581143 (21h21m59.853921242s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070588727 (21h21m59.853928826s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070817384 (21h21m59.854157483s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cCanaWd2hWD8vcQCophdGPopVihkjKSuUcB7ZyqqBcK1JTvSf): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070822854 (21h21m59.854162943s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070908688 (21h21m59.854248787s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YyuYGM7ttMRXybutoxVQBktQPes1XMce5fTs4BUfQVykeZtqT): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070913157 (21h21m59.854253256s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071162232 (21h21m59.854502341s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BL6sUUowTDCFa3Zu7Xv5SheJRFGzEHVd1yq3goxmvT5WggLg4): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.071169757 (21h21m59.854509866s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071187790 (21h21m59.854527889s) [seg_pos={0 0}, upload=20458]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 23:26:14.071292471 (21h21m59.85463257s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 23:26:14.071297069 (21h21m59.854637168s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 23:26:38.384275645 (21h22m24.167615734s) [upload=20458]: Done uploading segments...
May 25 23:26:38.384290073 (21h22m24.167630172s) [upload=20458]: Upload finished. err: <nil>
Duration: 4.109199108s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
13 out of 60 is failed (threshold=3) [X.X..X....X.........X.XX....X..X.X....................X.X..X] duration:
1m0.000786928s trace:
https://jaeger.dev.storj.io/trace/439368a58ab5c58
FAIL (2026-05-25 23:45:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000804362s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
5.897475205s trace:
https://jaeger.dev.storj.io/trace/154f04129c3ba223
FAIL (2026-05-25 23:44:54 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
H5UA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.669978445 (21h21m58.453318544s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670570324 (21h21m58.453910423s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670977932 (21h21m58.454318031s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671082672 (21h21m58.454422771s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671399046 (21h21m58.454739145s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671505018 (21h21m58.454845117s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671768601 (21h21m58.4551087s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671873380 (21h21m58.455213479s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672172411 (21h21m58.45551251s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672225192 (21h21m58.455565291s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672604115 (21h21m58.455944214s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672647878 (21h21m58.455987977s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672979411 (21h21m58.45631951s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673026311 (21h21m58.45636641s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673319009 (21h21m58.456659108s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:13.747141337 (21h21m59.530481436s) [upload=20458]: Next returned nil segment
May 25 23:26:13.747155513 (21h21m59.530495612s) [upload=20458]: Waiting for error group managing segments...
May 25 23:26:14.031706029 (21h21m59.815046128s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034098815 (21h21m59.817438914s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034662339 (21h21m59.818002439s) [noise=true , address=212.102.52.214:4362 , piece_id=ZSAFYLTPN3ESF7KDM6RXVTPIHI2O5BJ77WJQXDRGXEIM5IUTEVMQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034708708 (21h21m59.818048807s) [noise=true , address=212.102.52.217:4419 , piece_id=3IAS5VH7QJ2SJOMUCY2B4MCNYECWLHUH3AGH6MOGM4NS6KDAH5UA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035071280 (21h21m59.818411379s) [noise=true , address=212.102.52.217:4424 , piece_id=OHAYPIY666MWXWJTMWLPWTPE3RAXI6SG6NRFZRQS7VO4GKYCOECQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035492163 (21h21m59.818832272s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.036495778 (21h21m59.819835877s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.037130078 (21h21m59.820470177s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.040472716 (21h21m59.823812815s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.042158933 (21h21m59.825499032s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045074927 (21h21m59.828415026s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045309063 (21h21m59.828649162s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046163785 (21h21m59.829503874s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046705198 (21h21m59.830045297s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.062034446 (21h21m59.845374546s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070397062 (21h21m59.853737161s) [noise=true , address=212.102.52.213:4322 , piece_id=TW24WKT2HYX65MOG6ILC2GYJPHYS5VRAEKPWGNF3MMPVJYMAZPHQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070415798 (21h21m59.853755897s) [seg_pos={0 0}, upload=20458]: Satisfied cohort requirements after 16 pieces
May 25 23:26:14.070581143 (21h21m59.853921242s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070588727 (21h21m59.853928826s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070817384 (21h21m59.854157483s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cCanaWd2hWD8vcQCophdGPopVihkjKSuUcB7ZyqqBcK1JTvSf): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070822854 (21h21m59.854162943s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070908688 (21h21m59.854248787s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YyuYGM7ttMRXybutoxVQBktQPes1XMce5fTs4BUfQVykeZtqT): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070913157 (21h21m59.854253256s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071162232 (21h21m59.854502341s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BL6sUUowTDCFa3Zu7Xv5SheJRFGzEHVd1yq3goxmvT5WggLg4): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.071169757 (21h21m59.854509866s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071187790 (21h21m59.854527889s) [seg_pos={0 0}, upload=20458]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 23:26:14.071292471 (21h21m59.85463257s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 23:26:14.071297069 (21h21m59.854637168s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 23:26:38.384275645 (21h22m24.167615734s) [upload=20458]: Done uploading segments...
May 25 23:26:38.384290073 (21h22m24.167630172s) [upload=20458]: Upload finished. err: <nil>
Duration: 5.89749864s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
4.050551726s trace:
https://jaeger.dev.storj.io/trace/154f04129c3ba223
FAIL (2026-05-25 23:44:25 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
H5UA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.669978445 (21h21m58.453318544s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670570324 (21h21m58.453910423s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670977932 (21h21m58.454318031s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671082672 (21h21m58.454422771s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671399046 (21h21m58.454739145s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671505018 (21h21m58.454845117s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671768601 (21h21m58.4551087s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671873380 (21h21m58.455213479s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672172411 (21h21m58.45551251s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672225192 (21h21m58.455565291s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672604115 (21h21m58.455944214s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672647878 (21h21m58.455987977s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672979411 (21h21m58.45631951s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673026311 (21h21m58.45636641s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673319009 (21h21m58.456659108s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:13.747141337 (21h21m59.530481436s) [upload=20458]: Next returned nil segment
May 25 23:26:13.747155513 (21h21m59.530495612s) [upload=20458]: Waiting for error group managing segments...
May 25 23:26:14.031706029 (21h21m59.815046128s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034098815 (21h21m59.817438914s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034662339 (21h21m59.818002439s) [noise=true , address=212.102.52.214:4362 , piece_id=ZSAFYLTPN3ESF7KDM6RXVTPIHI2O5BJ77WJQXDRGXEIM5IUTEVMQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034708708 (21h21m59.818048807s) [noise=true , address=212.102.52.217:4419 , piece_id=3IAS5VH7QJ2SJOMUCY2B4MCNYECWLHUH3AGH6MOGM4NS6KDAH5UA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035071280 (21h21m59.818411379s) [noise=true , address=212.102.52.217:4424 , piece_id=OHAYPIY666MWXWJTMWLPWTPE3RAXI6SG6NRFZRQS7VO4GKYCOECQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035492163 (21h21m59.818832272s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.036495778 (21h21m59.819835877s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.037130078 (21h21m59.820470177s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.040472716 (21h21m59.823812815s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.042158933 (21h21m59.825499032s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045074927 (21h21m59.828415026s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045309063 (21h21m59.828649162s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046163785 (21h21m59.829503874s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046705198 (21h21m59.830045297s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.062034446 (21h21m59.845374546s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070397062 (21h21m59.853737161s) [noise=true , address=212.102.52.213:4322 , piece_id=TW24WKT2HYX65MOG6ILC2GYJPHYS5VRAEKPWGNF3MMPVJYMAZPHQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070415798 (21h21m59.853755897s) [seg_pos={0 0}, upload=20458]: Satisfied cohort requirements after 16 pieces
May 25 23:26:14.070581143 (21h21m59.853921242s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070588727 (21h21m59.853928826s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070817384 (21h21m59.854157483s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cCanaWd2hWD8vcQCophdGPopVihkjKSuUcB7ZyqqBcK1JTvSf): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070822854 (21h21m59.854162943s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070908688 (21h21m59.854248787s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YyuYGM7ttMRXybutoxVQBktQPes1XMce5fTs4BUfQVykeZtqT): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070913157 (21h21m59.854253256s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071162232 (21h21m59.854502341s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BL6sUUowTDCFa3Zu7Xv5SheJRFGzEHVd1yq3goxmvT5WggLg4): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.071169757 (21h21m59.854509866s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071187790 (21h21m59.854527889s) [seg_pos={0 0}, upload=20458]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 23:26:14.071292471 (21h21m59.85463257s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 23:26:14.071297069 (21h21m59.854637168s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 23:26:38.384275645 (21h22m24.167615734s) [upload=20458]: Done uploading segments...
May 25 23:26:38.384290073 (21h22m24.167630172s) [upload=20458]: Upload finished. err: <nil>
Duration: 4.050572907s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
4.188695659s trace:
https://jaeger.dev.storj.io/trace/154f04129c3ba223
FAIL (2026-05-25 23:43:44 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
H5UA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.669978445 (21h21m58.453318544s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670570324 (21h21m58.453910423s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670977932 (21h21m58.454318031s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671082672 (21h21m58.454422771s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671399046 (21h21m58.454739145s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671505018 (21h21m58.454845117s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671768601 (21h21m58.4551087s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671873380 (21h21m58.455213479s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672172411 (21h21m58.45551251s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672225192 (21h21m58.455565291s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672604115 (21h21m58.455944214s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672647878 (21h21m58.455987977s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672979411 (21h21m58.45631951s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673026311 (21h21m58.45636641s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673319009 (21h21m58.456659108s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:13.747141337 (21h21m59.530481436s) [upload=20458]: Next returned nil segment
May 25 23:26:13.747155513 (21h21m59.530495612s) [upload=20458]: Waiting for error group managing segments...
May 25 23:26:14.031706029 (21h21m59.815046128s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034098815 (21h21m59.817438914s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034662339 (21h21m59.818002439s) [noise=true , address=212.102.52.214:4362 , piece_id=ZSAFYLTPN3ESF7KDM6RXVTPIHI2O5BJ77WJQXDRGXEIM5IUTEVMQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034708708 (21h21m59.818048807s) [noise=true , address=212.102.52.217:4419 , piece_id=3IAS5VH7QJ2SJOMUCY2B4MCNYECWLHUH3AGH6MOGM4NS6KDAH5UA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035071280 (21h21m59.818411379s) [noise=true , address=212.102.52.217:4424 , piece_id=OHAYPIY666MWXWJTMWLPWTPE3RAXI6SG6NRFZRQS7VO4GKYCOECQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035492163 (21h21m59.818832272s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.036495778 (21h21m59.819835877s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.037130078 (21h21m59.820470177s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.040472716 (21h21m59.823812815s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.042158933 (21h21m59.825499032s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045074927 (21h21m59.828415026s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045309063 (21h21m59.828649162s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046163785 (21h21m59.829503874s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046705198 (21h21m59.830045297s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.062034446 (21h21m59.845374546s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070397062 (21h21m59.853737161s) [noise=true , address=212.102.52.213:4322 , piece_id=TW24WKT2HYX65MOG6ILC2GYJPHYS5VRAEKPWGNF3MMPVJYMAZPHQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070415798 (21h21m59.853755897s) [seg_pos={0 0}, upload=20458]: Satisfied cohort requirements after 16 pieces
May 25 23:26:14.070581143 (21h21m59.853921242s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070588727 (21h21m59.853928826s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070817384 (21h21m59.854157483s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cCanaWd2hWD8vcQCophdGPopVihkjKSuUcB7ZyqqBcK1JTvSf): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070822854 (21h21m59.854162943s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070908688 (21h21m59.854248787s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YyuYGM7ttMRXybutoxVQBktQPes1XMce5fTs4BUfQVykeZtqT): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070913157 (21h21m59.854253256s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071162232 (21h21m59.854502341s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BL6sUUowTDCFa3Zu7Xv5SheJRFGzEHVd1yq3goxmvT5WggLg4): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.071169757 (21h21m59.854509866s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071187790 (21h21m59.854527889s) [seg_pos={0 0}, upload=20458]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 23:26:14.071292471 (21h21m59.85463257s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 23:26:14.071297069 (21h21m59.854637168s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 23:26:38.384275645 (21h22m24.167615734s) [upload=20458]: Done uploading segments...
May 25 23:26:38.384290073 (21h22m24.167630172s) [upload=20458]: Upload finished. err: <nil>
Duration: 4.18872207s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
12 out of 60 is failed (threshold=3) [.X.X..X....X.........X.XX....X..X.X....................X.X..] duration:
6.723139524s trace:
https://jaeger.dev.storj.io/trace/491397fbd6a884f8
FAIL (2026-05-25 23:43:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.723161125s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
12 out of 60 is failed (threshold=3) [..X.X..X....X.........X.XX....X..X.X....................X.X.] duration:
11.040299899s trace:
https://jaeger.dev.storj.io/trace/491397fbd6a884f8
FAIL (2026-05-25 23:43:17 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 11.04031072s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
12 out of 60 is failed (threshold=3) [...X.X..X....X.........X.XX....X..X.X....................X.X] duration:
1m0.000942236s trace:
https://jaeger.dev.storj.io/trace/491397fbd6a884f8
FAIL (2026-05-25 23:43:05 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000961695s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
3.945128693s trace:
https://jaeger.dev.storj.io/trace/154f04129c3ba223
FAIL (2026-05-25 23:42:45 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
H5UA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.669978445 (21h21m58.453318544s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670570324 (21h21m58.453910423s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670977932 (21h21m58.454318031s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671082672 (21h21m58.454422771s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671399046 (21h21m58.454739145s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671505018 (21h21m58.454845117s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671768601 (21h21m58.4551087s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671873380 (21h21m58.455213479s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672172411 (21h21m58.45551251s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672225192 (21h21m58.455565291s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672604115 (21h21m58.455944214s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672647878 (21h21m58.455987977s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672979411 (21h21m58.45631951s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673026311 (21h21m58.45636641s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673319009 (21h21m58.456659108s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:13.747141337 (21h21m59.530481436s) [upload=20458]: Next returned nil segment
May 25 23:26:13.747155513 (21h21m59.530495612s) [upload=20458]: Waiting for error group managing segments...
May 25 23:26:14.031706029 (21h21m59.815046128s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034098815 (21h21m59.817438914s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034662339 (21h21m59.818002439s) [noise=true , address=212.102.52.214:4362 , piece_id=ZSAFYLTPN3ESF7KDM6RXVTPIHI2O5BJ77WJQXDRGXEIM5IUTEVMQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034708708 (21h21m59.818048807s) [noise=true , address=212.102.52.217:4419 , piece_id=3IAS5VH7QJ2SJOMUCY2B4MCNYECWLHUH3AGH6MOGM4NS6KDAH5UA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035071280 (21h21m59.818411379s) [noise=true , address=212.102.52.217:4424 , piece_id=OHAYPIY666MWXWJTMWLPWTPE3RAXI6SG6NRFZRQS7VO4GKYCOECQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035492163 (21h21m59.818832272s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.036495778 (21h21m59.819835877s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.037130078 (21h21m59.820470177s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.040472716 (21h21m59.823812815s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.042158933 (21h21m59.825499032s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045074927 (21h21m59.828415026s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045309063 (21h21m59.828649162s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046163785 (21h21m59.829503874s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046705198 (21h21m59.830045297s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.062034446 (21h21m59.845374546s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070397062 (21h21m59.853737161s) [noise=true , address=212.102.52.213:4322 , piece_id=TW24WKT2HYX65MOG6ILC2GYJPHYS5VRAEKPWGNF3MMPVJYMAZPHQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070415798 (21h21m59.853755897s) [seg_pos={0 0}, upload=20458]: Satisfied cohort requirements after 16 pieces
May 25 23:26:14.070581143 (21h21m59.853921242s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070588727 (21h21m59.853928826s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070817384 (21h21m59.854157483s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cCanaWd2hWD8vcQCophdGPopVihkjKSuUcB7ZyqqBcK1JTvSf): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070822854 (21h21m59.854162943s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070908688 (21h21m59.854248787s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YyuYGM7ttMRXybutoxVQBktQPes1XMce5fTs4BUfQVykeZtqT): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070913157 (21h21m59.854253256s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071162232 (21h21m59.854502341s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BL6sUUowTDCFa3Zu7Xv5SheJRFGzEHVd1yq3goxmvT5WggLg4): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.071169757 (21h21m59.854509866s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071187790 (21h21m59.854527889s) [seg_pos={0 0}, upload=20458]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 23:26:14.071292471 (21h21m59.85463257s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 23:26:14.071297069 (21h21m59.854637168s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 23:26:38.384275645 (21h22m24.167615734s) [upload=20458]: Done uploading segments...
May 25 23:26:38.384290073 (21h22m24.167630172s) [upload=20458]: Upload finished. err: <nil>
Duration: 3.945147389s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [X...........................................................] duration:
3.742482646s trace:
https://jaeger.dev.storj.io/trace/3643d66a94f1a334
FAIL (2026-05-25 23:42:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.74249491s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
4.155137008s trace:
https://jaeger.dev.storj.io/trace/154f04129c3ba223
FAIL (2026-05-25 23:42:04 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
H5UA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.669978445 (21h21m58.453318544s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670570324 (21h21m58.453910423s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670977932 (21h21m58.454318031s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671082672 (21h21m58.454422771s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671399046 (21h21m58.454739145s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671505018 (21h21m58.454845117s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671768601 (21h21m58.4551087s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671873380 (21h21m58.455213479s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672172411 (21h21m58.45551251s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672225192 (21h21m58.455565291s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672604115 (21h21m58.455944214s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672647878 (21h21m58.455987977s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672979411 (21h21m58.45631951s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673026311 (21h21m58.45636641s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673319009 (21h21m58.456659108s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:13.747141337 (21h21m59.530481436s) [upload=20458]: Next returned nil segment
May 25 23:26:13.747155513 (21h21m59.530495612s) [upload=20458]: Waiting for error group managing segments...
May 25 23:26:14.031706029 (21h21m59.815046128s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034098815 (21h21m59.817438914s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034662339 (21h21m59.818002439s) [noise=true , address=212.102.52.214:4362 , piece_id=ZSAFYLTPN3ESF7KDM6RXVTPIHI2O5BJ77WJQXDRGXEIM5IUTEVMQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034708708 (21h21m59.818048807s) [noise=true , address=212.102.52.217:4419 , piece_id=3IAS5VH7QJ2SJOMUCY2B4MCNYECWLHUH3AGH6MOGM4NS6KDAH5UA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035071280 (21h21m59.818411379s) [noise=true , address=212.102.52.217:4424 , piece_id=OHAYPIY666MWXWJTMWLPWTPE3RAXI6SG6NRFZRQS7VO4GKYCOECQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035492163 (21h21m59.818832272s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.036495778 (21h21m59.819835877s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.037130078 (21h21m59.820470177s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.040472716 (21h21m59.823812815s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.042158933 (21h21m59.825499032s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045074927 (21h21m59.828415026s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045309063 (21h21m59.828649162s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046163785 (21h21m59.829503874s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046705198 (21h21m59.830045297s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.062034446 (21h21m59.845374546s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070397062 (21h21m59.853737161s) [noise=true , address=212.102.52.213:4322 , piece_id=TW24WKT2HYX65MOG6ILC2GYJPHYS5VRAEKPWGNF3MMPVJYMAZPHQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070415798 (21h21m59.853755897s) [seg_pos={0 0}, upload=20458]: Satisfied cohort requirements after 16 pieces
May 25 23:26:14.070581143 (21h21m59.853921242s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070588727 (21h21m59.853928826s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070817384 (21h21m59.854157483s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cCanaWd2hWD8vcQCophdGPopVihkjKSuUcB7ZyqqBcK1JTvSf): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070822854 (21h21m59.854162943s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070908688 (21h21m59.854248787s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YyuYGM7ttMRXybutoxVQBktQPes1XMce5fTs4BUfQVykeZtqT): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070913157 (21h21m59.854253256s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071162232 (21h21m59.854502341s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BL6sUUowTDCFa3Zu7Xv5SheJRFGzEHVd1yq3goxmvT5WggLg4): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.071169757 (21h21m59.854509866s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071187790 (21h21m59.854527889s) [seg_pos={0 0}, upload=20458]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 23:26:14.071292471 (21h21m59.85463257s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 23:26:14.071297069 (21h21m59.854637168s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 23:26:38.384275645 (21h22m24.167615734s) [upload=20458]: Done uploading segments...
May 25 23:26:38.384290073 (21h22m24.167630172s) [upload=20458]: Upload finished. err: <nil>
Duration: 4.155158389s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.X..........................................................] duration:
3.769548329s trace:
https://jaeger.dev.storj.io/trace/3643d66a94f1a334
FAIL (2026-05-25 23:41:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.76956964s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
12 out of 60 is failed (threshold=3) [X...X.X..X....X.........X.XX....X..X.X....................X.] duration:
10.960605403s trace:
https://jaeger.dev.storj.io/trace/68d128af3abf7fc5
FAIL (2026-05-25 23:41:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 10.960618287s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
12 out of 60 is failed (threshold=3) [.X...X.X..X....X.........X.XX....X..X.X....................X] duration:
1m0.001322246s trace:
https://jaeger.dev.storj.io/trace/68d128af3abf7fc5
FAIL (2026-05-25 23:41:47 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.001330912s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..X.........................................................] duration:
3.774166549s trace:
https://jaeger.dev.storj.io/trace/3643d66a94f1a334
FAIL (2026-05-25 23:41:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.774183322s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
7.974038088s trace:
https://jaeger.dev.storj.io/trace/154f04129c3ba223
FAIL (2026-05-25 23:41:01 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
H5UA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.669978445 (21h21m58.453318544s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670570324 (21h21m58.453910423s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670977932 (21h21m58.454318031s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671082672 (21h21m58.454422771s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671399046 (21h21m58.454739145s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671505018 (21h21m58.454845117s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671768601 (21h21m58.4551087s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671873380 (21h21m58.455213479s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672172411 (21h21m58.45551251s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672225192 (21h21m58.455565291s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672604115 (21h21m58.455944214s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672647878 (21h21m58.455987977s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672979411 (21h21m58.45631951s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673026311 (21h21m58.45636641s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673319009 (21h21m58.456659108s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:13.747141337 (21h21m59.530481436s) [upload=20458]: Next returned nil segment
May 25 23:26:13.747155513 (21h21m59.530495612s) [upload=20458]: Waiting for error group managing segments...
May 25 23:26:14.031706029 (21h21m59.815046128s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034098815 (21h21m59.817438914s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034662339 (21h21m59.818002439s) [noise=true , address=212.102.52.214:4362 , piece_id=ZSAFYLTPN3ESF7KDM6RXVTPIHI2O5BJ77WJQXDRGXEIM5IUTEVMQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034708708 (21h21m59.818048807s) [noise=true , address=212.102.52.217:4419 , piece_id=3IAS5VH7QJ2SJOMUCY2B4MCNYECWLHUH3AGH6MOGM4NS6KDAH5UA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035071280 (21h21m59.818411379s) [noise=true , address=212.102.52.217:4424 , piece_id=OHAYPIY666MWXWJTMWLPWTPE3RAXI6SG6NRFZRQS7VO4GKYCOECQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035492163 (21h21m59.818832272s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.036495778 (21h21m59.819835877s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.037130078 (21h21m59.820470177s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.040472716 (21h21m59.823812815s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.042158933 (21h21m59.825499032s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045074927 (21h21m59.828415026s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045309063 (21h21m59.828649162s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046163785 (21h21m59.829503874s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046705198 (21h21m59.830045297s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.062034446 (21h21m59.845374546s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070397062 (21h21m59.853737161s) [noise=true , address=212.102.52.213:4322 , piece_id=TW24WKT2HYX65MOG6ILC2GYJPHYS5VRAEKPWGNF3MMPVJYMAZPHQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070415798 (21h21m59.853755897s) [seg_pos={0 0}, upload=20458]: Satisfied cohort requirements after 16 pieces
May 25 23:26:14.070581143 (21h21m59.853921242s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070588727 (21h21m59.853928826s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070817384 (21h21m59.854157483s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cCanaWd2hWD8vcQCophdGPopVihkjKSuUcB7ZyqqBcK1JTvSf): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070822854 (21h21m59.854162943s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070908688 (21h21m59.854248787s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YyuYGM7ttMRXybutoxVQBktQPes1XMce5fTs4BUfQVykeZtqT): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070913157 (21h21m59.854253256s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071162232 (21h21m59.854502341s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BL6sUUowTDCFa3Zu7Xv5SheJRFGzEHVd1yq3goxmvT5WggLg4): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.071169757 (21h21m59.854509866s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071187790 (21h21m59.854527889s) [seg_pos={0 0}, upload=20458]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 23:26:14.071292471 (21h21m59.85463257s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 23:26:14.071297069 (21h21m59.854637168s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 23:26:38.384275645 (21h22m24.167615734s) [upload=20458]: Done uploading segments...
May 25 23:26:38.384290073 (21h22m24.167630172s) [upload=20458]: Upload finished. err: <nil>
Duration: 7.974052324s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...X........................................................] duration:
3.73459212s trace:
https://jaeger.dev.storj.io/trace/3643d66a94f1a334
FAIL (2026-05-25 23:40:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.734611278s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
11 out of 60 is failed (threshold=3) [..X...X.X..X....X.........X.XX....X..X.X....................] duration:
58.851921869s trace:
https://jaeger.dev.storj.io/trace/37dbd4f0eda78722
FAIL (2026-05-25 23:40:47 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 58.851944812s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
4.066226459s trace:
https://jaeger.dev.storj.io/trace/154f04129c3ba223
FAIL (2026-05-25 23:40:13 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
H5UA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.669978445 (21h21m58.453318544s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670570324 (21h21m58.453910423s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670977932 (21h21m58.454318031s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671082672 (21h21m58.454422771s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671399046 (21h21m58.454739145s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671505018 (21h21m58.454845117s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671768601 (21h21m58.4551087s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671873380 (21h21m58.455213479s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672172411 (21h21m58.45551251s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672225192 (21h21m58.455565291s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672604115 (21h21m58.455944214s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672647878 (21h21m58.455987977s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672979411 (21h21m58.45631951s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673026311 (21h21m58.45636641s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673319009 (21h21m58.456659108s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:13.747141337 (21h21m59.530481436s) [upload=20458]: Next returned nil segment
May 25 23:26:13.747155513 (21h21m59.530495612s) [upload=20458]: Waiting for error group managing segments...
May 25 23:26:14.031706029 (21h21m59.815046128s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034098815 (21h21m59.817438914s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034662339 (21h21m59.818002439s) [noise=true , address=212.102.52.214:4362 , piece_id=ZSAFYLTPN3ESF7KDM6RXVTPIHI2O5BJ77WJQXDRGXEIM5IUTEVMQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034708708 (21h21m59.818048807s) [noise=true , address=212.102.52.217:4419 , piece_id=3IAS5VH7QJ2SJOMUCY2B4MCNYECWLHUH3AGH6MOGM4NS6KDAH5UA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035071280 (21h21m59.818411379s) [noise=true , address=212.102.52.217:4424 , piece_id=OHAYPIY666MWXWJTMWLPWTPE3RAXI6SG6NRFZRQS7VO4GKYCOECQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035492163 (21h21m59.818832272s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.036495778 (21h21m59.819835877s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.037130078 (21h21m59.820470177s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.040472716 (21h21m59.823812815s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.042158933 (21h21m59.825499032s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045074927 (21h21m59.828415026s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045309063 (21h21m59.828649162s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046163785 (21h21m59.829503874s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046705198 (21h21m59.830045297s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.062034446 (21h21m59.845374546s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070397062 (21h21m59.853737161s) [noise=true , address=212.102.52.213:4322 , piece_id=TW24WKT2HYX65MOG6ILC2GYJPHYS5VRAEKPWGNF3MMPVJYMAZPHQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070415798 (21h21m59.853755897s) [seg_pos={0 0}, upload=20458]: Satisfied cohort requirements after 16 pieces
May 25 23:26:14.070581143 (21h21m59.853921242s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070588727 (21h21m59.853928826s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070817384 (21h21m59.854157483s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cCanaWd2hWD8vcQCophdGPopVihkjKSuUcB7ZyqqBcK1JTvSf): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070822854 (21h21m59.854162943s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070908688 (21h21m59.854248787s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YyuYGM7ttMRXybutoxVQBktQPes1XMce5fTs4BUfQVykeZtqT): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070913157 (21h21m59.854253256s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071162232 (21h21m59.854502341s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BL6sUUowTDCFa3Zu7Xv5SheJRFGzEHVd1yq3goxmvT5WggLg4): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.071169757 (21h21m59.854509866s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071187790 (21h21m59.854527889s) [seg_pos={0 0}, upload=20458]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 23:26:14.071292471 (21h21m59.85463257s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 23:26:14.071297069 (21h21m59.854637168s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 23:26:38.384275645 (21h22m24.167615734s) [upload=20458]: Done uploading segments...
May 25 23:26:38.384290073 (21h22m24.167630172s) [upload=20458]: Upload finished. err: <nil>
Duration: 4.066248702s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [....X.......................................................] duration:
3.840938544s trace:
https://jaeger.dev.storj.io/trace/3643d66a94f1a334
FAIL (2026-05-25 23:39:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.840960726s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
3.945551183s trace:
https://jaeger.dev.storj.io/trace/154f04129c3ba223
FAIL (2026-05-25 23:39:30 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
H5UA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.669978445 (21h21m58.453318544s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670570324 (21h21m58.453910423s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670977932 (21h21m58.454318031s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671082672 (21h21m58.454422771s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671399046 (21h21m58.454739145s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671505018 (21h21m58.454845117s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671768601 (21h21m58.4551087s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671873380 (21h21m58.455213479s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672172411 (21h21m58.45551251s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672225192 (21h21m58.455565291s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672604115 (21h21m58.455944214s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672647878 (21h21m58.455987977s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672979411 (21h21m58.45631951s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673026311 (21h21m58.45636641s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673319009 (21h21m58.456659108s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:13.747141337 (21h21m59.530481436s) [upload=20458]: Next returned nil segment
May 25 23:26:13.747155513 (21h21m59.530495612s) [upload=20458]: Waiting for error group managing segments...
May 25 23:26:14.031706029 (21h21m59.815046128s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034098815 (21h21m59.817438914s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034662339 (21h21m59.818002439s) [noise=true , address=212.102.52.214:4362 , piece_id=ZSAFYLTPN3ESF7KDM6RXVTPIHI2O5BJ77WJQXDRGXEIM5IUTEVMQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034708708 (21h21m59.818048807s) [noise=true , address=212.102.52.217:4419 , piece_id=3IAS5VH7QJ2SJOMUCY2B4MCNYECWLHUH3AGH6MOGM4NS6KDAH5UA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035071280 (21h21m59.818411379s) [noise=true , address=212.102.52.217:4424 , piece_id=OHAYPIY666MWXWJTMWLPWTPE3RAXI6SG6NRFZRQS7VO4GKYCOECQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035492163 (21h21m59.818832272s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.036495778 (21h21m59.819835877s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.037130078 (21h21m59.820470177s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.040472716 (21h21m59.823812815s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.042158933 (21h21m59.825499032s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045074927 (21h21m59.828415026s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045309063 (21h21m59.828649162s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046163785 (21h21m59.829503874s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046705198 (21h21m59.830045297s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.062034446 (21h21m59.845374546s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070397062 (21h21m59.853737161s) [noise=true , address=212.102.52.213:4322 , piece_id=TW24WKT2HYX65MOG6ILC2GYJPHYS5VRAEKPWGNF3MMPVJYMAZPHQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070415798 (21h21m59.853755897s) [seg_pos={0 0}, upload=20458]: Satisfied cohort requirements after 16 pieces
May 25 23:26:14.070581143 (21h21m59.853921242s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070588727 (21h21m59.853928826s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070817384 (21h21m59.854157483s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cCanaWd2hWD8vcQCophdGPopVihkjKSuUcB7ZyqqBcK1JTvSf): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070822854 (21h21m59.854162943s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070908688 (21h21m59.854248787s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YyuYGM7ttMRXybutoxVQBktQPes1XMce5fTs4BUfQVykeZtqT): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070913157 (21h21m59.854253256s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071162232 (21h21m59.854502341s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BL6sUUowTDCFa3Zu7Xv5SheJRFGzEHVd1yq3goxmvT5WggLg4): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.071169757 (21h21m59.854509866s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071187790 (21h21m59.854527889s) [seg_pos={0 0}, upload=20458]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 23:26:14.071292471 (21h21m59.85463257s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 23:26:14.071297069 (21h21m59.854637168s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 23:26:38.384275645 (21h22m24.167615734s) [upload=20458]: Done uploading segments...
May 25 23:26:38.384290073 (21h22m24.167630172s) [upload=20458]: Upload finished. err: <nil>
Duration: 3.945566172s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
5.490639735s trace:
https://jaeger.dev.storj.io/trace/154f04129c3ba223
FAIL (2026-05-25 23:39:05 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
H5UA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.669978445 (21h21m58.453318544s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670570324 (21h21m58.453910423s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670977932 (21h21m58.454318031s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671082672 (21h21m58.454422771s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671399046 (21h21m58.454739145s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671505018 (21h21m58.454845117s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671768601 (21h21m58.4551087s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671873380 (21h21m58.455213479s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672172411 (21h21m58.45551251s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672225192 (21h21m58.455565291s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672604115 (21h21m58.455944214s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672647878 (21h21m58.455987977s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672979411 (21h21m58.45631951s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673026311 (21h21m58.45636641s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673319009 (21h21m58.456659108s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:13.747141337 (21h21m59.530481436s) [upload=20458]: Next returned nil segment
May 25 23:26:13.747155513 (21h21m59.530495612s) [upload=20458]: Waiting for error group managing segments...
May 25 23:26:14.031706029 (21h21m59.815046128s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034098815 (21h21m59.817438914s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034662339 (21h21m59.818002439s) [noise=true , address=212.102.52.214:4362 , piece_id=ZSAFYLTPN3ESF7KDM6RXVTPIHI2O5BJ77WJQXDRGXEIM5IUTEVMQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034708708 (21h21m59.818048807s) [noise=true , address=212.102.52.217:4419 , piece_id=3IAS5VH7QJ2SJOMUCY2B4MCNYECWLHUH3AGH6MOGM4NS6KDAH5UA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035071280 (21h21m59.818411379s) [noise=true , address=212.102.52.217:4424 , piece_id=OHAYPIY666MWXWJTMWLPWTPE3RAXI6SG6NRFZRQS7VO4GKYCOECQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035492163 (21h21m59.818832272s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.036495778 (21h21m59.819835877s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.037130078 (21h21m59.820470177s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.040472716 (21h21m59.823812815s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.042158933 (21h21m59.825499032s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045074927 (21h21m59.828415026s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045309063 (21h21m59.828649162s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046163785 (21h21m59.829503874s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046705198 (21h21m59.830045297s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.062034446 (21h21m59.845374546s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070397062 (21h21m59.853737161s) [noise=true , address=212.102.52.213:4322 , piece_id=TW24WKT2HYX65MOG6ILC2GYJPHYS5VRAEKPWGNF3MMPVJYMAZPHQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070415798 (21h21m59.853755897s) [seg_pos={0 0}, upload=20458]: Satisfied cohort requirements after 16 pieces
May 25 23:26:14.070581143 (21h21m59.853921242s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070588727 (21h21m59.853928826s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070817384 (21h21m59.854157483s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cCanaWd2hWD8vcQCophdGPopVihkjKSuUcB7ZyqqBcK1JTvSf): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070822854 (21h21m59.854162943s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070908688 (21h21m59.854248787s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YyuYGM7ttMRXybutoxVQBktQPes1XMce5fTs4BUfQVykeZtqT): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070913157 (21h21m59.854253256s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071162232 (21h21m59.854502341s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BL6sUUowTDCFa3Zu7Xv5SheJRFGzEHVd1yq3goxmvT5WggLg4): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.071169757 (21h21m59.854509866s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071187790 (21h21m59.854527889s) [seg_pos={0 0}, upload=20458]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 23:26:14.071292471 (21h21m59.85463257s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 23:26:14.071297069 (21h21m59.854637168s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 23:26:38.384275645 (21h22m24.167615734s) [upload=20458]: Done uploading segments...
May 25 23:26:38.384290073 (21h22m24.167630172s) [upload=20458]: Upload finished. err: <nil>
Duration: 5.49065792s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.....X......................................................] duration:
4.145627221s trace:
https://jaeger.dev.storj.io/trace/3643d66a94f1a334
FAIL (2026-05-25 23:38:50 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.145673662s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
11 out of 60 is failed (threshold=3) [...X...X.X..X....X.........X.XX....X..X.X...................] duration:
6.543572617s trace:
https://jaeger.dev.storj.io/trace/37dbd4f0eda78722
FAIL (2026-05-25 23:38:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.54359512s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
4.38814125s trace:
https://jaeger.dev.storj.io/trace/154f04129c3ba223
FAIL (2026-05-25 23:38:11 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
H5UA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.669978445 (21h21m58.453318544s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670570324 (21h21m58.453910423s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670977932 (21h21m58.454318031s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671082672 (21h21m58.454422771s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671399046 (21h21m58.454739145s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671505018 (21h21m58.454845117s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671768601 (21h21m58.4551087s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671873380 (21h21m58.455213479s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672172411 (21h21m58.45551251s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672225192 (21h21m58.455565291s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672604115 (21h21m58.455944214s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672647878 (21h21m58.455987977s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672979411 (21h21m58.45631951s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673026311 (21h21m58.45636641s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673319009 (21h21m58.456659108s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:13.747141337 (21h21m59.530481436s) [upload=20458]: Next returned nil segment
May 25 23:26:13.747155513 (21h21m59.530495612s) [upload=20458]: Waiting for error group managing segments...
May 25 23:26:14.031706029 (21h21m59.815046128s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034098815 (21h21m59.817438914s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034662339 (21h21m59.818002439s) [noise=true , address=212.102.52.214:4362 , piece_id=ZSAFYLTPN3ESF7KDM6RXVTPIHI2O5BJ77WJQXDRGXEIM5IUTEVMQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034708708 (21h21m59.818048807s) [noise=true , address=212.102.52.217:4419 , piece_id=3IAS5VH7QJ2SJOMUCY2B4MCNYECWLHUH3AGH6MOGM4NS6KDAH5UA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035071280 (21h21m59.818411379s) [noise=true , address=212.102.52.217:4424 , piece_id=OHAYPIY666MWXWJTMWLPWTPE3RAXI6SG6NRFZRQS7VO4GKYCOECQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035492163 (21h21m59.818832272s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.036495778 (21h21m59.819835877s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.037130078 (21h21m59.820470177s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.040472716 (21h21m59.823812815s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.042158933 (21h21m59.825499032s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045074927 (21h21m59.828415026s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045309063 (21h21m59.828649162s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046163785 (21h21m59.829503874s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046705198 (21h21m59.830045297s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.062034446 (21h21m59.845374546s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070397062 (21h21m59.853737161s) [noise=true , address=212.102.52.213:4322 , piece_id=TW24WKT2HYX65MOG6ILC2GYJPHYS5VRAEKPWGNF3MMPVJYMAZPHQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070415798 (21h21m59.853755897s) [seg_pos={0 0}, upload=20458]: Satisfied cohort requirements after 16 pieces
May 25 23:26:14.070581143 (21h21m59.853921242s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070588727 (21h21m59.853928826s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070817384 (21h21m59.854157483s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cCanaWd2hWD8vcQCophdGPopVihkjKSuUcB7ZyqqBcK1JTvSf): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070822854 (21h21m59.854162943s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070908688 (21h21m59.854248787s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YyuYGM7ttMRXybutoxVQBktQPes1XMce5fTs4BUfQVykeZtqT): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070913157 (21h21m59.854253256s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071162232 (21h21m59.854502341s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BL6sUUowTDCFa3Zu7Xv5SheJRFGzEHVd1yq3goxmvT5WggLg4): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.071169757 (21h21m59.854509866s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071187790 (21h21m59.854527889s) [seg_pos={0 0}, upload=20458]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 23:26:14.071292471 (21h21m59.85463257s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 23:26:14.071297069 (21h21m59.854637168s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 23:26:38.384275645 (21h22m24.167615734s) [upload=20458]: Done uploading segments...
May 25 23:26:38.384290073 (21h22m24.167630172s) [upload=20458]: Upload finished. err: <nil>
Duration: 4.388164453s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [......X.....................................................] duration:
3.812532532s trace:
https://jaeger.dev.storj.io/trace/3643d66a94f1a334
FAIL (2026-05-25 23:38:06 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.812546618s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
12 out of 60 is failed (threshold=3) [X...X...X.X..X....X.........X.XX....X..X.X..................] duration:
6.48219149s trace:
https://jaeger.dev.storj.io/trace/37dbd4f0eda78722
FAIL (2026-05-25 23:37:50 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.482240916s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
5.11186156s trace:
https://jaeger.dev.storj.io/trace/154f04129c3ba223
FAIL (2026-05-25 23:37:33 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
H5UA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.669978445 (21h21m58.453318544s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670570324 (21h21m58.453910423s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670977932 (21h21m58.454318031s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671082672 (21h21m58.454422771s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671399046 (21h21m58.454739145s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671505018 (21h21m58.454845117s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671768601 (21h21m58.4551087s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671873380 (21h21m58.455213479s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672172411 (21h21m58.45551251s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672225192 (21h21m58.455565291s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672604115 (21h21m58.455944214s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672647878 (21h21m58.455987977s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672979411 (21h21m58.45631951s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673026311 (21h21m58.45636641s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673319009 (21h21m58.456659108s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:13.747141337 (21h21m59.530481436s) [upload=20458]: Next returned nil segment
May 25 23:26:13.747155513 (21h21m59.530495612s) [upload=20458]: Waiting for error group managing segments...
May 25 23:26:14.031706029 (21h21m59.815046128s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034098815 (21h21m59.817438914s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034662339 (21h21m59.818002439s) [noise=true , address=212.102.52.214:4362 , piece_id=ZSAFYLTPN3ESF7KDM6RXVTPIHI2O5BJ77WJQXDRGXEIM5IUTEVMQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034708708 (21h21m59.818048807s) [noise=true , address=212.102.52.217:4419 , piece_id=3IAS5VH7QJ2SJOMUCY2B4MCNYECWLHUH3AGH6MOGM4NS6KDAH5UA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035071280 (21h21m59.818411379s) [noise=true , address=212.102.52.217:4424 , piece_id=OHAYPIY666MWXWJTMWLPWTPE3RAXI6SG6NRFZRQS7VO4GKYCOECQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035492163 (21h21m59.818832272s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.036495778 (21h21m59.819835877s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.037130078 (21h21m59.820470177s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.040472716 (21h21m59.823812815s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.042158933 (21h21m59.825499032s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045074927 (21h21m59.828415026s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045309063 (21h21m59.828649162s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046163785 (21h21m59.829503874s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046705198 (21h21m59.830045297s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.062034446 (21h21m59.845374546s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070397062 (21h21m59.853737161s) [noise=true , address=212.102.52.213:4322 , piece_id=TW24WKT2HYX65MOG6ILC2GYJPHYS5VRAEKPWGNF3MMPVJYMAZPHQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070415798 (21h21m59.853755897s) [seg_pos={0 0}, upload=20458]: Satisfied cohort requirements after 16 pieces
May 25 23:26:14.070581143 (21h21m59.853921242s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070588727 (21h21m59.853928826s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070817384 (21h21m59.854157483s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cCanaWd2hWD8vcQCophdGPopVihkjKSuUcB7ZyqqBcK1JTvSf): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070822854 (21h21m59.854162943s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070908688 (21h21m59.854248787s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YyuYGM7ttMRXybutoxVQBktQPes1XMce5fTs4BUfQVykeZtqT): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070913157 (21h21m59.854253256s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071162232 (21h21m59.854502341s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BL6sUUowTDCFa3Zu7Xv5SheJRFGzEHVd1yq3goxmvT5WggLg4): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.071169757 (21h21m59.854509866s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071187790 (21h21m59.854527889s) [seg_pos={0 0}, upload=20458]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 23:26:14.071292471 (21h21m59.85463257s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 23:26:14.071297069 (21h21m59.854637168s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 23:26:38.384275645 (21h22m24.167615734s) [upload=20458]: Done uploading segments...
May 25 23:26:38.384290073 (21h22m24.167630172s) [upload=20458]: Upload finished. err: <nil>
Duration: 5.111883964s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.......X....................................................] duration:
3.739996173s trace:
https://jaeger.dev.storj.io/trace/3643d66a94f1a334
FAIL (2026-05-25 23:37:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.740016894s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
12 out of 60 is failed (threshold=3) [.X...X...X.X..X....X.........X.XX....X..X.X.................] duration:
6.523412224s trace:
https://jaeger.dev.storj.io/trace/37dbd4f0eda78722
FAIL (2026-05-25 23:37:02 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.523423946s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [........X...................................................] duration:
3.825197142s trace:
https://jaeger.dev.storj.io/trace/3643d66a94f1a334
FAIL (2026-05-25 23:36:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.825211049s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
12 out of 60 is failed (threshold=3) [..X...X...X.X..X....X.........X.XX....X..X.X................] duration:
54.505197791s trace:
https://jaeger.dev.storj.io/trace/37dbd4f0eda78722
FAIL (2026-05-25 23:36:55 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 54.50530752s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
4.003441187s trace:
https://jaeger.dev.storj.io/trace/154f04129c3ba223
FAIL (2026-05-25 23:36:40 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
H5UA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.669978445 (21h21m58.453318544s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670570324 (21h21m58.453910423s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670977932 (21h21m58.454318031s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671082672 (21h21m58.454422771s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671399046 (21h21m58.454739145s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671505018 (21h21m58.454845117s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671768601 (21h21m58.4551087s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671873380 (21h21m58.455213479s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672172411 (21h21m58.45551251s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672225192 (21h21m58.455565291s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672604115 (21h21m58.455944214s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672647878 (21h21m58.455987977s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672979411 (21h21m58.45631951s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673026311 (21h21m58.45636641s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673319009 (21h21m58.456659108s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:13.747141337 (21h21m59.530481436s) [upload=20458]: Next returned nil segment
May 25 23:26:13.747155513 (21h21m59.530495612s) [upload=20458]: Waiting for error group managing segments...
May 25 23:26:14.031706029 (21h21m59.815046128s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034098815 (21h21m59.817438914s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034662339 (21h21m59.818002439s) [noise=true , address=212.102.52.214:4362 , piece_id=ZSAFYLTPN3ESF7KDM6RXVTPIHI2O5BJ77WJQXDRGXEIM5IUTEVMQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034708708 (21h21m59.818048807s) [noise=true , address=212.102.52.217:4419 , piece_id=3IAS5VH7QJ2SJOMUCY2B4MCNYECWLHUH3AGH6MOGM4NS6KDAH5UA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035071280 (21h21m59.818411379s) [noise=true , address=212.102.52.217:4424 , piece_id=OHAYPIY666MWXWJTMWLPWTPE3RAXI6SG6NRFZRQS7VO4GKYCOECQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035492163 (21h21m59.818832272s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.036495778 (21h21m59.819835877s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.037130078 (21h21m59.820470177s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.040472716 (21h21m59.823812815s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.042158933 (21h21m59.825499032s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045074927 (21h21m59.828415026s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045309063 (21h21m59.828649162s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046163785 (21h21m59.829503874s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046705198 (21h21m59.830045297s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.062034446 (21h21m59.845374546s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070397062 (21h21m59.853737161s) [noise=true , address=212.102.52.213:4322 , piece_id=TW24WKT2HYX65MOG6ILC2GYJPHYS5VRAEKPWGNF3MMPVJYMAZPHQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070415798 (21h21m59.853755897s) [seg_pos={0 0}, upload=20458]: Satisfied cohort requirements after 16 pieces
May 25 23:26:14.070581143 (21h21m59.853921242s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070588727 (21h21m59.853928826s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070817384 (21h21m59.854157483s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cCanaWd2hWD8vcQCophdGPopVihkjKSuUcB7ZyqqBcK1JTvSf): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070822854 (21h21m59.854162943s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070908688 (21h21m59.854248787s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YyuYGM7ttMRXybutoxVQBktQPes1XMce5fTs4BUfQVykeZtqT): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070913157 (21h21m59.854253256s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071162232 (21h21m59.854502341s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BL6sUUowTDCFa3Zu7Xv5SheJRFGzEHVd1yq3goxmvT5WggLg4): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.071169757 (21h21m59.854509866s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071187790 (21h21m59.854527889s) [seg_pos={0 0}, upload=20458]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 23:26:14.071292471 (21h21m59.85463257s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 23:26:14.071297069 (21h21m59.854637168s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 23:26:38.384275645 (21h22m24.167615734s) [upload=20458]: Done uploading segments...
May 25 23:26:38.384290073 (21h22m24.167630172s) [upload=20458]: Upload finished. err: <nil>
Duration: 4.003464171s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.........X..................................................] duration:
3.565852601s trace:
https://jaeger.dev.storj.io/trace/3643d66a94f1a334
FAIL (2026-05-25 23:35:50 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.565871437s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
4.004468714s trace:
https://jaeger.dev.storj.io/trace/154f04129c3ba223
FAIL (2026-05-25 23:35:40 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
H5UA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.669978445 (21h21m58.453318544s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670570324 (21h21m58.453910423s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670977932 (21h21m58.454318031s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671082672 (21h21m58.454422771s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671399046 (21h21m58.454739145s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671505018 (21h21m58.454845117s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671768601 (21h21m58.4551087s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671873380 (21h21m58.455213479s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672172411 (21h21m58.45551251s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672225192 (21h21m58.455565291s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672604115 (21h21m58.455944214s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672647878 (21h21m58.455987977s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672979411 (21h21m58.45631951s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673026311 (21h21m58.45636641s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673319009 (21h21m58.456659108s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:13.747141337 (21h21m59.530481436s) [upload=20458]: Next returned nil segment
May 25 23:26:13.747155513 (21h21m59.530495612s) [upload=20458]: Waiting for error group managing segments...
May 25 23:26:14.031706029 (21h21m59.815046128s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034098815 (21h21m59.817438914s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034662339 (21h21m59.818002439s) [noise=true , address=212.102.52.214:4362 , piece_id=ZSAFYLTPN3ESF7KDM6RXVTPIHI2O5BJ77WJQXDRGXEIM5IUTEVMQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034708708 (21h21m59.818048807s) [noise=true , address=212.102.52.217:4419 , piece_id=3IAS5VH7QJ2SJOMUCY2B4MCNYECWLHUH3AGH6MOGM4NS6KDAH5UA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035071280 (21h21m59.818411379s) [noise=true , address=212.102.52.217:4424 , piece_id=OHAYPIY666MWXWJTMWLPWTPE3RAXI6SG6NRFZRQS7VO4GKYCOECQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035492163 (21h21m59.818832272s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.036495778 (21h21m59.819835877s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.037130078 (21h21m59.820470177s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.040472716 (21h21m59.823812815s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.042158933 (21h21m59.825499032s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045074927 (21h21m59.828415026s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045309063 (21h21m59.828649162s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046163785 (21h21m59.829503874s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046705198 (21h21m59.830045297s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.062034446 (21h21m59.845374546s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070397062 (21h21m59.853737161s) [noise=true , address=212.102.52.213:4322 , piece_id=TW24WKT2HYX65MOG6ILC2GYJPHYS5VRAEKPWGNF3MMPVJYMAZPHQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070415798 (21h21m59.853755897s) [seg_pos={0 0}, upload=20458]: Satisfied cohort requirements after 16 pieces
May 25 23:26:14.070581143 (21h21m59.853921242s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070588727 (21h21m59.853928826s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070817384 (21h21m59.854157483s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cCanaWd2hWD8vcQCophdGPopVihkjKSuUcB7ZyqqBcK1JTvSf): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070822854 (21h21m59.854162943s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070908688 (21h21m59.854248787s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YyuYGM7ttMRXybutoxVQBktQPes1XMce5fTs4BUfQVykeZtqT): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070913157 (21h21m59.854253256s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071162232 (21h21m59.854502341s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BL6sUUowTDCFa3Zu7Xv5SheJRFGzEHVd1yq3goxmvT5WggLg4): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.071169757 (21h21m59.854509866s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071187790 (21h21m59.854527889s) [seg_pos={0 0}, upload=20458]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 23:26:14.071292471 (21h21m59.85463257s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 23:26:14.071297069 (21h21m59.854637168s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 23:26:38.384275645 (21h22m24.167615734s) [upload=20458]: Done uploading segments...
May 25 23:26:38.384290073 (21h22m24.167630172s) [upload=20458]: Upload finished. err: <nil>
Duration: 4.004511806s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
12 out of 60 is failed (threshold=3) [...X...X...X.X..X....X.........X.XX....X..X.X...............] duration:
6.477950639s trace:
https://jaeger.dev.storj.io/trace/37dbd4f0eda78722
FAIL (2026-05-25 23:35:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.477965758s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
6.394313331s trace:
https://jaeger.dev.storj.io/trace/154f04129c3ba223
FAIL (2026-05-25 23:35:25 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
H5UA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.669978445 (21h21m58.453318544s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670570324 (21h21m58.453910423s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670977932 (21h21m58.454318031s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671082672 (21h21m58.454422771s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671399046 (21h21m58.454739145s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671505018 (21h21m58.454845117s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671768601 (21h21m58.4551087s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671873380 (21h21m58.455213479s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672172411 (21h21m58.45551251s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672225192 (21h21m58.455565291s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672604115 (21h21m58.455944214s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672647878 (21h21m58.455987977s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672979411 (21h21m58.45631951s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673026311 (21h21m58.45636641s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673319009 (21h21m58.456659108s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:13.747141337 (21h21m59.530481436s) [upload=20458]: Next returned nil segment
May 25 23:26:13.747155513 (21h21m59.530495612s) [upload=20458]: Waiting for error group managing segments...
May 25 23:26:14.031706029 (21h21m59.815046128s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034098815 (21h21m59.817438914s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034662339 (21h21m59.818002439s) [noise=true , address=212.102.52.214:4362 , piece_id=ZSAFYLTPN3ESF7KDM6RXVTPIHI2O5BJ77WJQXDRGXEIM5IUTEVMQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034708708 (21h21m59.818048807s) [noise=true , address=212.102.52.217:4419 , piece_id=3IAS5VH7QJ2SJOMUCY2B4MCNYECWLHUH3AGH6MOGM4NS6KDAH5UA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035071280 (21h21m59.818411379s) [noise=true , address=212.102.52.217:4424 , piece_id=OHAYPIY666MWXWJTMWLPWTPE3RAXI6SG6NRFZRQS7VO4GKYCOECQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035492163 (21h21m59.818832272s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.036495778 (21h21m59.819835877s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.037130078 (21h21m59.820470177s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.040472716 (21h21m59.823812815s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.042158933 (21h21m59.825499032s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045074927 (21h21m59.828415026s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045309063 (21h21m59.828649162s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046163785 (21h21m59.829503874s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046705198 (21h21m59.830045297s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.062034446 (21h21m59.845374546s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070397062 (21h21m59.853737161s) [noise=true , address=212.102.52.213:4322 , piece_id=TW24WKT2HYX65MOG6ILC2GYJPHYS5VRAEKPWGNF3MMPVJYMAZPHQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070415798 (21h21m59.853755897s) [seg_pos={0 0}, upload=20458]: Satisfied cohort requirements after 16 pieces
May 25 23:26:14.070581143 (21h21m59.853921242s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070588727 (21h21m59.853928826s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070817384 (21h21m59.854157483s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cCanaWd2hWD8vcQCophdGPopVihkjKSuUcB7ZyqqBcK1JTvSf): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070822854 (21h21m59.854162943s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070908688 (21h21m59.854248787s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YyuYGM7ttMRXybutoxVQBktQPes1XMce5fTs4BUfQVykeZtqT): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070913157 (21h21m59.854253256s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071162232 (21h21m59.854502341s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BL6sUUowTDCFa3Zu7Xv5SheJRFGzEHVd1yq3goxmvT5WggLg4): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.071169757 (21h21m59.854509866s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071187790 (21h21m59.854527889s) [seg_pos={0 0}, upload=20458]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 23:26:14.071292471 (21h21m59.85463257s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 23:26:14.071297069 (21h21m59.854637168s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 23:26:38.384275645 (21h22m24.167615734s) [upload=20458]: Done uploading segments...
May 25 23:26:38.384290073 (21h22m24.167630172s) [upload=20458]: Upload finished. err: <nil>
Duration: 6.394331946s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
13 out of 60 is failed (threshold=3) [X...X...X...X.X..X....X.........X.XX....X..X.X..............] duration:
6.669309171s trace:
https://jaeger.dev.storj.io/trace/37dbd4f0eda78722
FAIL (2026-05-25 23:35:12 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.669326826s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..........X.................................................] duration:
4.499027847s trace:
https://jaeger.dev.storj.io/trace/3643d66a94f1a334
FAIL (2026-05-25 23:35:06 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.499050902s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
4.14670785s trace:
https://jaeger.dev.storj.io/trace/154f04129c3ba223
FAIL (2026-05-25 23:35:05 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
H5UA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.669978445 (21h21m58.453318544s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670570324 (21h21m58.453910423s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670977932 (21h21m58.454318031s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671082672 (21h21m58.454422771s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671399046 (21h21m58.454739145s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671505018 (21h21m58.454845117s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671768601 (21h21m58.4551087s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671873380 (21h21m58.455213479s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672172411 (21h21m58.45551251s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672225192 (21h21m58.455565291s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672604115 (21h21m58.455944214s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672647878 (21h21m58.455987977s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672979411 (21h21m58.45631951s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673026311 (21h21m58.45636641s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673319009 (21h21m58.456659108s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:13.747141337 (21h21m59.530481436s) [upload=20458]: Next returned nil segment
May 25 23:26:13.747155513 (21h21m59.530495612s) [upload=20458]: Waiting for error group managing segments...
May 25 23:26:14.031706029 (21h21m59.815046128s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034098815 (21h21m59.817438914s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034662339 (21h21m59.818002439s) [noise=true , address=212.102.52.214:4362 , piece_id=ZSAFYLTPN3ESF7KDM6RXVTPIHI2O5BJ77WJQXDRGXEIM5IUTEVMQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034708708 (21h21m59.818048807s) [noise=true , address=212.102.52.217:4419 , piece_id=3IAS5VH7QJ2SJOMUCY2B4MCNYECWLHUH3AGH6MOGM4NS6KDAH5UA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035071280 (21h21m59.818411379s) [noise=true , address=212.102.52.217:4424 , piece_id=OHAYPIY666MWXWJTMWLPWTPE3RAXI6SG6NRFZRQS7VO4GKYCOECQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035492163 (21h21m59.818832272s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.036495778 (21h21m59.819835877s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.037130078 (21h21m59.820470177s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.040472716 (21h21m59.823812815s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.042158933 (21h21m59.825499032s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045074927 (21h21m59.828415026s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045309063 (21h21m59.828649162s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046163785 (21h21m59.829503874s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046705198 (21h21m59.830045297s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.062034446 (21h21m59.845374546s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070397062 (21h21m59.853737161s) [noise=true , address=212.102.52.213:4322 , piece_id=TW24WKT2HYX65MOG6ILC2GYJPHYS5VRAEKPWGNF3MMPVJYMAZPHQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070415798 (21h21m59.853755897s) [seg_pos={0 0}, upload=20458]: Satisfied cohort requirements after 16 pieces
May 25 23:26:14.070581143 (21h21m59.853921242s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070588727 (21h21m59.853928826s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070817384 (21h21m59.854157483s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cCanaWd2hWD8vcQCophdGPopVihkjKSuUcB7ZyqqBcK1JTvSf): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070822854 (21h21m59.854162943s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070908688 (21h21m59.854248787s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YyuYGM7ttMRXybutoxVQBktQPes1XMce5fTs4BUfQVykeZtqT): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070913157 (21h21m59.854253256s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071162232 (21h21m59.854502341s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BL6sUUowTDCFa3Zu7Xv5SheJRFGzEHVd1yq3goxmvT5WggLg4): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.071169757 (21h21m59.854509866s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071187790 (21h21m59.854527889s) [seg_pos={0 0}, upload=20458]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 23:26:14.071292471 (21h21m59.85463257s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 23:26:14.071297069 (21h21m59.854637168s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 23:26:38.384275645 (21h22m24.167615734s) [upload=20458]: Done uploading segments...
May 25 23:26:38.384290073 (21h22m24.167630172s) [upload=20458]: Upload finished. err: <nil>
Duration: 4.14672864s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...........X................................................] duration:
3.964800198s trace:
https://jaeger.dev.storj.io/trace/3643d66a94f1a334
FAIL (2026-05-25 23:34:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.965053622s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
13 out of 60 is failed (threshold=3) [.X...X...X...X.X..X....X.........X.XX....X..X.X.............] duration:
6.839368489s trace:
https://jaeger.dev.storj.io/trace/37dbd4f0eda78722
FAIL (2026-05-25 23:34:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.83939003s
core
uplink
us-select-1
us1-placement-20
|