Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [....................................................X.XX.XX.] duration:
4.751038215s trace:
https://jaeger.dev.storj.io/trace/573c194341e02876
FAIL (2026-04-17 17:06:34 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.751059786s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
6.237499535s trace:
https://jaeger.dev.storj.io/trace/1c80c1acf9bc1e4e
FAIL (2026-04-17 17:05:52 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.237523761s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [.....................................................X.XX.XX] duration:
5.000803093s trace:
https://jaeger.dev.storj.io/trace/573c194341e02876
FAIL (2026-04-17 17:05:45 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000828682s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
6.027606243s trace:
https://jaeger.dev.storj.io/trace/1c80c1acf9bc1e4e
FAIL (2026-04-17 17:04:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.027948474s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [......................................................X.XX.X] duration:
5.000932473s trace:
https://jaeger.dev.storj.io/trace/36269449378c0687
FAIL (2026-04-17 17:04:34 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000947793s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [.......................................................X.XX.] duration:
3.437079229s trace:
https://jaeger.dev.storj.io/trace/76a42048c5ff82a2
FAIL (2026-04-17 17:03:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.437101703s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
5.91544372s trace:
https://jaeger.dev.storj.io/trace/1c80c1acf9bc1e4e
FAIL (2026-04-17 17:03:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.915466344s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [........................................................X.XX] duration:
5.001349135s trace:
https://jaeger.dev.storj.io/trace/76a42048c5ff82a2
FAIL (2026-04-17 17:03:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001360837s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
6.202689181s trace:
https://jaeger.dev.storj.io/trace/1c80c1acf9bc1e4e
FAIL (2026-04-17 17:03:02 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.202910128s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
6.186364232s trace:
https://jaeger.dev.storj.io/trace/1c80c1acf9bc1e4e
FAIL (2026-04-17 17:02:46 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.186386084s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.........................................................X.X] duration:
5.002032024s trace:
https://jaeger.dev.storj.io/trace/3be551057193e775
FAIL (2026-04-17 17:02:40 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.002065528s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..........................................................X.] duration:
5.000470903s trace:
https://jaeger.dev.storj.io/trace/1b42fa1d514528cf
FAIL (2026-04-17 17:02:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000493757s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
5.9429028s trace:
https://jaeger.dev.storj.io/trace/1c80c1acf9bc1e4e
FAIL (2026-04-17 17:01:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.942926898s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...........................................................X] duration:
5.001276879s trace:
https://jaeger.dev.storj.io/trace/1b42fa1d514528cf
FAIL (2026-04-17 17:01:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001295965s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
6.815109778s trace:
https://jaeger.dev.storj.io/trace/1c80c1acf9bc1e4e
FAIL (2026-04-17 17:01:29 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.815130558s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
6.84666783s trace:
https://jaeger.dev.storj.io/trace/1c80c1acf9bc1e4e
FAIL (2026-04-17 17:00:34 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.846694863s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
6.724182048s trace:
https://jaeger.dev.storj.io/trace/1c80c1acf9bc1e4e
FAIL (2026-04-17 16:59:32 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.724208561s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
4.555995819s trace:
https://jaeger.dev.storj.io/trace/7f8a8e8c290cfce4
FAIL (2026-04-17 16:59:27 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: context deadline exceeded
======LOGS======
}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468778967 (240h28m13.288014991s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468982109 (240h28m13.288218133s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469034200 (240h28m13.288270224s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469278090 (240h28m13.288514114s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469317958 (240h28m13.288553982s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469533453 (240h28m13.288769477s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469783256 (240h28m13.28901928s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470149613 (240h28m13.289385637s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470407480 (240h28m13.289643504s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:27.287464596 (240h28m14.10670062s) [upload=230887]: Next returned nil segment
Apr 17 16:14:27.287476399 (240h28m14.106712423s) [upload=230887]: Waiting for error group managing segments...
Apr 17 16:14:27.667935875 (240h28m14.487171899s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.669691854 (240h28m14.488927878s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.672696102 (240h28m14.491932116s) [noise=true , address=185.167.204.206:4044 , piece_id=PV5G7HUDQCIZRBRC4U4QBSXDPYMS6OKELUO6T376WSCHIJEKSSHA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.673873083 (240h28m14.493109108s) [noise=true , address=185.167.204.205:4014 , piece_id=NZ2QB3PRHEREH7L3JN6QWWKEUSCCZJPX4KX5TOBBTEFI7V77CLDQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674467470 (240h28m14.493703494s) [noise=true , address=185.167.204.206:4041 , piece_id=5KYRFUXZKMCXVC6I2XF3MWFFFCBKRDEQU32J46PELU2K7ITHDJCQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674861982 (240h28m14.494098006s) [noise=true , address=185.167.204.205:4023 , piece_id=GX2SDFS6H5G3OV5K3JPMX4AFDJJWQAZZMJWCFCDLIQJ6J2EEGBRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675219832 (240h28m14.494455856s) [noise=true , address=185.167.204.206:4062 , piece_id=JLJJYF2HITUHLDWANNO7TANPR2B55XBEPI73RGIEQWQAUXVAUOLQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675769814 (240h28m14.495005838s) [noise=true , address=185.167.204.205:4034 , piece_id=2YLRAMZ4EUMTBIAOP47JESS2CNUE72WD2X7WNS3G5SNXTSIKZE6Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.679234931 (240h28m14.498470955s) [noise=true , address=185.167.204.205:4005 , piece_id=K5BMMNJJLPMJJ37OERBC7A2Y2GS6O7M5SKYFQSVUHWFWFQVC52SQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.680568836 (240h28m14.49980485s) [noise=true , address=185.167.204.205:4033 , piece_id=BVOQ7VXKB65CRU3M3APCYJ3NKFG6HG6UGNNU7UKI4CJYCYL5HY2Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.681606178 (240h28m14.500842202s) [noise=true , address=185.167.204.205:4004 , piece_id=UATYU6M64CYLYFO7ZGZZTMY6TUJM34WGYM5U4FKE2EADZ3WFHHXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.682849166 (240h28m14.50208519s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.683007592 (240h28m14.502243616s) [noise=true , address=185.167.204.205:4003 , piece_id=OAJS25SOIZDYBJK43WRYDDMO3OYR6P6NCE7O37KN7X2PICJNOZXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.684400130 (240h28m14.503636154s) [noise=true , address=185.167.204.205:4036 , piece_id=RMXRKSJM5DRMODM2CAZBXTKUMFTG4B4O26TMKC233WIOTRSR6KRQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.686095452 (240h28m14.505331466s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.747734833 (240h28m14.566970857s) [noise=true , address=81.171.14.70:4046 , piece_id=4NMIQBBHLZX3AZRNTOHSJ2LM5A26OV66FL7HENHJTGJPFZPDZO4A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:28.990034383 (240h28m15.809270407s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.104032302 (240h28m15.923268326s) [noise=true , address=89.149.201.147:4011 , piece_id=VJZIZ4X7SKEQCXU665VXZLLQ4IF5I2S5CAY45IZSIUG5X4JKPZFA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.227923806 (240h28m16.04715983s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.327415309 (240h28m16.146651333s) [noise=true , address=81.171.14.70:4044 , piece_id=I2ML43UPUBAZWL6LNXEXOXVCO4VMHQULA6LIZ6QWSDES4DC4RMYA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:30.946412145 (240h28m17.765648159s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.310420672 (240h28m18.129656686s) [noise=true , address=89.149.201.147:4018 , piece_id=QQ4VAJ2AKLWWRV4FSPUMYB4CW64GZNVDBCHHF2FLLPUFKFOELHMA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.578914230 (240h28m18.398150274s) [noise=true , address=81.171.14.70:4030 , piece_id=RGRX2LD7VJUKZUI7OTHL54N4TCX6BT27SG2656CHNK3C4USIM3XA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:32.696287178 (240h28m19.515523202s) [noise=true , address=89.149.201.147:4004 , piece_id=YYIMUGDQSCZBIYNQRDQHY3FCHGDUQ7NYMD2P77N2KJTMHVCC5NLA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.122355826 (240h28m19.94159185s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622613706 (240h28m20.44184973s) [noise=true , address=81.171.14.70:4045 , piece_id=OXF6AHOJRIBSLIM2AHDMWHJELCXJJOK4FDUCZX4R2IWKSFIT4GTQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622631460 (240h28m20.441867484s) [seg_pos={0 0}, upload=230887]: Satisfied cohort requirements after 26 pieces
Apr 17 16:14:33.622880831 (240h28m20.442116865s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC): protocol: read tcp 188.166.225.170:33320->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622889879 (240h28m20.442125913s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622919847 (240h28m20.442155871s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jcAHt22bETT27CCXJ8ZaaYxxoKBonBYsFcoq6A34DVVjvhQAN): protocol: read tcp 188.166.225.170:44412->81.171.14.70:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622924756 (240h28m20.44216078s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622936359 (240h28m20.442172383s) [seg_pos={0 0}, upload=230887]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 17 16:14:33.622990012 (240h28m20.442226036s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 17 16:14:33.622993769 (240h28m20.442229793s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 17 16:14:33.857773622 (240h28m20.677009646s) [upload=230887]: Done uploading segments...
Apr 17 16:14:33.857791256 (240h28m20.67702728s) [upload=230887]: Upload finished. err: <nil>
Duration: 4.556071326s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
6.711127525s trace:
https://jaeger.dev.storj.io/trace/1c80c1acf9bc1e4e
FAIL (2026-04-17 16:58:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.711178483s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
4.506459793s trace:
https://jaeger.dev.storj.io/trace/7f8a8e8c290cfce4
FAIL (2026-04-17 16:58:13 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: context deadline exceeded
======LOGS======
}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468778967 (240h28m13.288014991s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468982109 (240h28m13.288218133s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469034200 (240h28m13.288270224s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469278090 (240h28m13.288514114s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469317958 (240h28m13.288553982s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469533453 (240h28m13.288769477s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469783256 (240h28m13.28901928s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470149613 (240h28m13.289385637s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470407480 (240h28m13.289643504s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:27.287464596 (240h28m14.10670062s) [upload=230887]: Next returned nil segment
Apr 17 16:14:27.287476399 (240h28m14.106712423s) [upload=230887]: Waiting for error group managing segments...
Apr 17 16:14:27.667935875 (240h28m14.487171899s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.669691854 (240h28m14.488927878s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.672696102 (240h28m14.491932116s) [noise=true , address=185.167.204.206:4044 , piece_id=PV5G7HUDQCIZRBRC4U4QBSXDPYMS6OKELUO6T376WSCHIJEKSSHA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.673873083 (240h28m14.493109108s) [noise=true , address=185.167.204.205:4014 , piece_id=NZ2QB3PRHEREH7L3JN6QWWKEUSCCZJPX4KX5TOBBTEFI7V77CLDQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674467470 (240h28m14.493703494s) [noise=true , address=185.167.204.206:4041 , piece_id=5KYRFUXZKMCXVC6I2XF3MWFFFCBKRDEQU32J46PELU2K7ITHDJCQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674861982 (240h28m14.494098006s) [noise=true , address=185.167.204.205:4023 , piece_id=GX2SDFS6H5G3OV5K3JPMX4AFDJJWQAZZMJWCFCDLIQJ6J2EEGBRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675219832 (240h28m14.494455856s) [noise=true , address=185.167.204.206:4062 , piece_id=JLJJYF2HITUHLDWANNO7TANPR2B55XBEPI73RGIEQWQAUXVAUOLQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675769814 (240h28m14.495005838s) [noise=true , address=185.167.204.205:4034 , piece_id=2YLRAMZ4EUMTBIAOP47JESS2CNUE72WD2X7WNS3G5SNXTSIKZE6Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.679234931 (240h28m14.498470955s) [noise=true , address=185.167.204.205:4005 , piece_id=K5BMMNJJLPMJJ37OERBC7A2Y2GS6O7M5SKYFQSVUHWFWFQVC52SQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.680568836 (240h28m14.49980485s) [noise=true , address=185.167.204.205:4033 , piece_id=BVOQ7VXKB65CRU3M3APCYJ3NKFG6HG6UGNNU7UKI4CJYCYL5HY2Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.681606178 (240h28m14.500842202s) [noise=true , address=185.167.204.205:4004 , piece_id=UATYU6M64CYLYFO7ZGZZTMY6TUJM34WGYM5U4FKE2EADZ3WFHHXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.682849166 (240h28m14.50208519s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.683007592 (240h28m14.502243616s) [noise=true , address=185.167.204.205:4003 , piece_id=OAJS25SOIZDYBJK43WRYDDMO3OYR6P6NCE7O37KN7X2PICJNOZXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.684400130 (240h28m14.503636154s) [noise=true , address=185.167.204.205:4036 , piece_id=RMXRKSJM5DRMODM2CAZBXTKUMFTG4B4O26TMKC233WIOTRSR6KRQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.686095452 (240h28m14.505331466s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.747734833 (240h28m14.566970857s) [noise=true , address=81.171.14.70:4046 , piece_id=4NMIQBBHLZX3AZRNTOHSJ2LM5A26OV66FL7HENHJTGJPFZPDZO4A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:28.990034383 (240h28m15.809270407s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.104032302 (240h28m15.923268326s) [noise=true , address=89.149.201.147:4011 , piece_id=VJZIZ4X7SKEQCXU665VXZLLQ4IF5I2S5CAY45IZSIUG5X4JKPZFA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.227923806 (240h28m16.04715983s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.327415309 (240h28m16.146651333s) [noise=true , address=81.171.14.70:4044 , piece_id=I2ML43UPUBAZWL6LNXEXOXVCO4VMHQULA6LIZ6QWSDES4DC4RMYA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:30.946412145 (240h28m17.765648159s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.310420672 (240h28m18.129656686s) [noise=true , address=89.149.201.147:4018 , piece_id=QQ4VAJ2AKLWWRV4FSPUMYB4CW64GZNVDBCHHF2FLLPUFKFOELHMA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.578914230 (240h28m18.398150274s) [noise=true , address=81.171.14.70:4030 , piece_id=RGRX2LD7VJUKZUI7OTHL54N4TCX6BT27SG2656CHNK3C4USIM3XA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:32.696287178 (240h28m19.515523202s) [noise=true , address=89.149.201.147:4004 , piece_id=YYIMUGDQSCZBIYNQRDQHY3FCHGDUQ7NYMD2P77N2KJTMHVCC5NLA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.122355826 (240h28m19.94159185s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622613706 (240h28m20.44184973s) [noise=true , address=81.171.14.70:4045 , piece_id=OXF6AHOJRIBSLIM2AHDMWHJELCXJJOK4FDUCZX4R2IWKSFIT4GTQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622631460 (240h28m20.441867484s) [seg_pos={0 0}, upload=230887]: Satisfied cohort requirements after 26 pieces
Apr 17 16:14:33.622880831 (240h28m20.442116865s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC): protocol: read tcp 188.166.225.170:33320->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622889879 (240h28m20.442125913s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622919847 (240h28m20.442155871s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jcAHt22bETT27CCXJ8ZaaYxxoKBonBYsFcoq6A34DVVjvhQAN): protocol: read tcp 188.166.225.170:44412->81.171.14.70:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622924756 (240h28m20.44216078s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622936359 (240h28m20.442172383s) [seg_pos={0 0}, upload=230887]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 17 16:14:33.622990012 (240h28m20.442226036s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 17 16:14:33.622993769 (240h28m20.442229793s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 17 16:14:33.857773622 (240h28m20.677009646s) [upload=230887]: Done uploading segments...
Apr 17 16:14:33.857791256 (240h28m20.67702728s) [upload=230887]: Upload finished. err: <nil>
Duration: 4.506489971s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
7.766586899s trace:
https://jaeger.dev.storj.io/trace/1c80c1acf9bc1e4e
FAIL (2026-04-17 16:58:12 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.766680139s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
6.659908274s trace:
https://jaeger.dev.storj.io/trace/1c80c1acf9bc1e4e
FAIL (2026-04-17 16:57:50 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.65993714s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
5.099054516s trace:
https://jaeger.dev.storj.io/trace/7f8a8e8c290cfce4
FAIL (2026-04-17 16:57:25 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: context deadline exceeded
======LOGS======
}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468778967 (240h28m13.288014991s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468982109 (240h28m13.288218133s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469034200 (240h28m13.288270224s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469278090 (240h28m13.288514114s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469317958 (240h28m13.288553982s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469533453 (240h28m13.288769477s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469783256 (240h28m13.28901928s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470149613 (240h28m13.289385637s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470407480 (240h28m13.289643504s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:27.287464596 (240h28m14.10670062s) [upload=230887]: Next returned nil segment
Apr 17 16:14:27.287476399 (240h28m14.106712423s) [upload=230887]: Waiting for error group managing segments...
Apr 17 16:14:27.667935875 (240h28m14.487171899s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.669691854 (240h28m14.488927878s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.672696102 (240h28m14.491932116s) [noise=true , address=185.167.204.206:4044 , piece_id=PV5G7HUDQCIZRBRC4U4QBSXDPYMS6OKELUO6T376WSCHIJEKSSHA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.673873083 (240h28m14.493109108s) [noise=true , address=185.167.204.205:4014 , piece_id=NZ2QB3PRHEREH7L3JN6QWWKEUSCCZJPX4KX5TOBBTEFI7V77CLDQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674467470 (240h28m14.493703494s) [noise=true , address=185.167.204.206:4041 , piece_id=5KYRFUXZKMCXVC6I2XF3MWFFFCBKRDEQU32J46PELU2K7ITHDJCQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674861982 (240h28m14.494098006s) [noise=true , address=185.167.204.205:4023 , piece_id=GX2SDFS6H5G3OV5K3JPMX4AFDJJWQAZZMJWCFCDLIQJ6J2EEGBRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675219832 (240h28m14.494455856s) [noise=true , address=185.167.204.206:4062 , piece_id=JLJJYF2HITUHLDWANNO7TANPR2B55XBEPI73RGIEQWQAUXVAUOLQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675769814 (240h28m14.495005838s) [noise=true , address=185.167.204.205:4034 , piece_id=2YLRAMZ4EUMTBIAOP47JESS2CNUE72WD2X7WNS3G5SNXTSIKZE6Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.679234931 (240h28m14.498470955s) [noise=true , address=185.167.204.205:4005 , piece_id=K5BMMNJJLPMJJ37OERBC7A2Y2GS6O7M5SKYFQSVUHWFWFQVC52SQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.680568836 (240h28m14.49980485s) [noise=true , address=185.167.204.205:4033 , piece_id=BVOQ7VXKB65CRU3M3APCYJ3NKFG6HG6UGNNU7UKI4CJYCYL5HY2Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.681606178 (240h28m14.500842202s) [noise=true , address=185.167.204.205:4004 , piece_id=UATYU6M64CYLYFO7ZGZZTMY6TUJM34WGYM5U4FKE2EADZ3WFHHXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.682849166 (240h28m14.50208519s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.683007592 (240h28m14.502243616s) [noise=true , address=185.167.204.205:4003 , piece_id=OAJS25SOIZDYBJK43WRYDDMO3OYR6P6NCE7O37KN7X2PICJNOZXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.684400130 (240h28m14.503636154s) [noise=true , address=185.167.204.205:4036 , piece_id=RMXRKSJM5DRMODM2CAZBXTKUMFTG4B4O26TMKC233WIOTRSR6KRQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.686095452 (240h28m14.505331466s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.747734833 (240h28m14.566970857s) [noise=true , address=81.171.14.70:4046 , piece_id=4NMIQBBHLZX3AZRNTOHSJ2LM5A26OV66FL7HENHJTGJPFZPDZO4A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:28.990034383 (240h28m15.809270407s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.104032302 (240h28m15.923268326s) [noise=true , address=89.149.201.147:4011 , piece_id=VJZIZ4X7SKEQCXU665VXZLLQ4IF5I2S5CAY45IZSIUG5X4JKPZFA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.227923806 (240h28m16.04715983s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.327415309 (240h28m16.146651333s) [noise=true , address=81.171.14.70:4044 , piece_id=I2ML43UPUBAZWL6LNXEXOXVCO4VMHQULA6LIZ6QWSDES4DC4RMYA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:30.946412145 (240h28m17.765648159s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.310420672 (240h28m18.129656686s) [noise=true , address=89.149.201.147:4018 , piece_id=QQ4VAJ2AKLWWRV4FSPUMYB4CW64GZNVDBCHHF2FLLPUFKFOELHMA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.578914230 (240h28m18.398150274s) [noise=true , address=81.171.14.70:4030 , piece_id=RGRX2LD7VJUKZUI7OTHL54N4TCX6BT27SG2656CHNK3C4USIM3XA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:32.696287178 (240h28m19.515523202s) [noise=true , address=89.149.201.147:4004 , piece_id=YYIMUGDQSCZBIYNQRDQHY3FCHGDUQ7NYMD2P77N2KJTMHVCC5NLA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.122355826 (240h28m19.94159185s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622613706 (240h28m20.44184973s) [noise=true , address=81.171.14.70:4045 , piece_id=OXF6AHOJRIBSLIM2AHDMWHJELCXJJOK4FDUCZX4R2IWKSFIT4GTQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622631460 (240h28m20.441867484s) [seg_pos={0 0}, upload=230887]: Satisfied cohort requirements after 26 pieces
Apr 17 16:14:33.622880831 (240h28m20.442116865s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC): protocol: read tcp 188.166.225.170:33320->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622889879 (240h28m20.442125913s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622919847 (240h28m20.442155871s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jcAHt22bETT27CCXJ8ZaaYxxoKBonBYsFcoq6A34DVVjvhQAN): protocol: read tcp 188.166.225.170:44412->81.171.14.70:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622924756 (240h28m20.44216078s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622936359 (240h28m20.442172383s) [seg_pos={0 0}, upload=230887]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 17 16:14:33.622990012 (240h28m20.442226036s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 17 16:14:33.622993769 (240h28m20.442229793s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 17 16:14:33.857773622 (240h28m20.677009646s) [upload=230887]: Done uploading segments...
Apr 17 16:14:33.857791256 (240h28m20.67702728s) [upload=230887]: Upload finished. err: <nil>
Duration: 5.099081368s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
6.779307121s trace:
https://jaeger.dev.storj.io/trace/1c80c1acf9bc1e4e
FAIL (2026-04-17 16:57:17 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.779330426s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
4.650388612s trace:
https://jaeger.dev.storj.io/trace/7f8a8e8c290cfce4
FAIL (2026-04-17 16:56:38 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: context deadline exceeded
======LOGS======
}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468778967 (240h28m13.288014991s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468982109 (240h28m13.288218133s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469034200 (240h28m13.288270224s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469278090 (240h28m13.288514114s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469317958 (240h28m13.288553982s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469533453 (240h28m13.288769477s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469783256 (240h28m13.28901928s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470149613 (240h28m13.289385637s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470407480 (240h28m13.289643504s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:27.287464596 (240h28m14.10670062s) [upload=230887]: Next returned nil segment
Apr 17 16:14:27.287476399 (240h28m14.106712423s) [upload=230887]: Waiting for error group managing segments...
Apr 17 16:14:27.667935875 (240h28m14.487171899s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.669691854 (240h28m14.488927878s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.672696102 (240h28m14.491932116s) [noise=true , address=185.167.204.206:4044 , piece_id=PV5G7HUDQCIZRBRC4U4QBSXDPYMS6OKELUO6T376WSCHIJEKSSHA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.673873083 (240h28m14.493109108s) [noise=true , address=185.167.204.205:4014 , piece_id=NZ2QB3PRHEREH7L3JN6QWWKEUSCCZJPX4KX5TOBBTEFI7V77CLDQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674467470 (240h28m14.493703494s) [noise=true , address=185.167.204.206:4041 , piece_id=5KYRFUXZKMCXVC6I2XF3MWFFFCBKRDEQU32J46PELU2K7ITHDJCQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674861982 (240h28m14.494098006s) [noise=true , address=185.167.204.205:4023 , piece_id=GX2SDFS6H5G3OV5K3JPMX4AFDJJWQAZZMJWCFCDLIQJ6J2EEGBRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675219832 (240h28m14.494455856s) [noise=true , address=185.167.204.206:4062 , piece_id=JLJJYF2HITUHLDWANNO7TANPR2B55XBEPI73RGIEQWQAUXVAUOLQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675769814 (240h28m14.495005838s) [noise=true , address=185.167.204.205:4034 , piece_id=2YLRAMZ4EUMTBIAOP47JESS2CNUE72WD2X7WNS3G5SNXTSIKZE6Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.679234931 (240h28m14.498470955s) [noise=true , address=185.167.204.205:4005 , piece_id=K5BMMNJJLPMJJ37OERBC7A2Y2GS6O7M5SKYFQSVUHWFWFQVC52SQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.680568836 (240h28m14.49980485s) [noise=true , address=185.167.204.205:4033 , piece_id=BVOQ7VXKB65CRU3M3APCYJ3NKFG6HG6UGNNU7UKI4CJYCYL5HY2Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.681606178 (240h28m14.500842202s) [noise=true , address=185.167.204.205:4004 , piece_id=UATYU6M64CYLYFO7ZGZZTMY6TUJM34WGYM5U4FKE2EADZ3WFHHXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.682849166 (240h28m14.50208519s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.683007592 (240h28m14.502243616s) [noise=true , address=185.167.204.205:4003 , piece_id=OAJS25SOIZDYBJK43WRYDDMO3OYR6P6NCE7O37KN7X2PICJNOZXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.684400130 (240h28m14.503636154s) [noise=true , address=185.167.204.205:4036 , piece_id=RMXRKSJM5DRMODM2CAZBXTKUMFTG4B4O26TMKC233WIOTRSR6KRQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.686095452 (240h28m14.505331466s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.747734833 (240h28m14.566970857s) [noise=true , address=81.171.14.70:4046 , piece_id=4NMIQBBHLZX3AZRNTOHSJ2LM5A26OV66FL7HENHJTGJPFZPDZO4A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:28.990034383 (240h28m15.809270407s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.104032302 (240h28m15.923268326s) [noise=true , address=89.149.201.147:4011 , piece_id=VJZIZ4X7SKEQCXU665VXZLLQ4IF5I2S5CAY45IZSIUG5X4JKPZFA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.227923806 (240h28m16.04715983s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.327415309 (240h28m16.146651333s) [noise=true , address=81.171.14.70:4044 , piece_id=I2ML43UPUBAZWL6LNXEXOXVCO4VMHQULA6LIZ6QWSDES4DC4RMYA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:30.946412145 (240h28m17.765648159s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.310420672 (240h28m18.129656686s) [noise=true , address=89.149.201.147:4018 , piece_id=QQ4VAJ2AKLWWRV4FSPUMYB4CW64GZNVDBCHHF2FLLPUFKFOELHMA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.578914230 (240h28m18.398150274s) [noise=true , address=81.171.14.70:4030 , piece_id=RGRX2LD7VJUKZUI7OTHL54N4TCX6BT27SG2656CHNK3C4USIM3XA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:32.696287178 (240h28m19.515523202s) [noise=true , address=89.149.201.147:4004 , piece_id=YYIMUGDQSCZBIYNQRDQHY3FCHGDUQ7NYMD2P77N2KJTMHVCC5NLA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.122355826 (240h28m19.94159185s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622613706 (240h28m20.44184973s) [noise=true , address=81.171.14.70:4045 , piece_id=OXF6AHOJRIBSLIM2AHDMWHJELCXJJOK4FDUCZX4R2IWKSFIT4GTQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622631460 (240h28m20.441867484s) [seg_pos={0 0}, upload=230887]: Satisfied cohort requirements after 26 pieces
Apr 17 16:14:33.622880831 (240h28m20.442116865s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC): protocol: read tcp 188.166.225.170:33320->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622889879 (240h28m20.442125913s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622919847 (240h28m20.442155871s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jcAHt22bETT27CCXJ8ZaaYxxoKBonBYsFcoq6A34DVVjvhQAN): protocol: read tcp 188.166.225.170:44412->81.171.14.70:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622924756 (240h28m20.44216078s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622936359 (240h28m20.442172383s) [seg_pos={0 0}, upload=230887]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 17 16:14:33.622990012 (240h28m20.442226036s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 17 16:14:33.622993769 (240h28m20.442229793s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 17 16:14:33.857773622 (240h28m20.677009646s) [upload=230887]: Done uploading segments...
Apr 17 16:14:33.857791256 (240h28m20.67702728s) [upload=230887]: Upload finished. err: <nil>
Duration: 4.65041813s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
7.150462006s trace:
https://jaeger.dev.storj.io/trace/1c80c1acf9bc1e4e
FAIL (2026-04-17 16:56:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.150482126s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
4.139851009s trace:
https://jaeger.dev.storj.io/trace/7f8a8e8c290cfce4
FAIL (2026-04-17 16:56:22 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: context deadline exceeded
======LOGS======
}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468778967 (240h28m13.288014991s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468982109 (240h28m13.288218133s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469034200 (240h28m13.288270224s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469278090 (240h28m13.288514114s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469317958 (240h28m13.288553982s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469533453 (240h28m13.288769477s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469783256 (240h28m13.28901928s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470149613 (240h28m13.289385637s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470407480 (240h28m13.289643504s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:27.287464596 (240h28m14.10670062s) [upload=230887]: Next returned nil segment
Apr 17 16:14:27.287476399 (240h28m14.106712423s) [upload=230887]: Waiting for error group managing segments...
Apr 17 16:14:27.667935875 (240h28m14.487171899s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.669691854 (240h28m14.488927878s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.672696102 (240h28m14.491932116s) [noise=true , address=185.167.204.206:4044 , piece_id=PV5G7HUDQCIZRBRC4U4QBSXDPYMS6OKELUO6T376WSCHIJEKSSHA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.673873083 (240h28m14.493109108s) [noise=true , address=185.167.204.205:4014 , piece_id=NZ2QB3PRHEREH7L3JN6QWWKEUSCCZJPX4KX5TOBBTEFI7V77CLDQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674467470 (240h28m14.493703494s) [noise=true , address=185.167.204.206:4041 , piece_id=5KYRFUXZKMCXVC6I2XF3MWFFFCBKRDEQU32J46PELU2K7ITHDJCQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674861982 (240h28m14.494098006s) [noise=true , address=185.167.204.205:4023 , piece_id=GX2SDFS6H5G3OV5K3JPMX4AFDJJWQAZZMJWCFCDLIQJ6J2EEGBRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675219832 (240h28m14.494455856s) [noise=true , address=185.167.204.206:4062 , piece_id=JLJJYF2HITUHLDWANNO7TANPR2B55XBEPI73RGIEQWQAUXVAUOLQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675769814 (240h28m14.495005838s) [noise=true , address=185.167.204.205:4034 , piece_id=2YLRAMZ4EUMTBIAOP47JESS2CNUE72WD2X7WNS3G5SNXTSIKZE6Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.679234931 (240h28m14.498470955s) [noise=true , address=185.167.204.205:4005 , piece_id=K5BMMNJJLPMJJ37OERBC7A2Y2GS6O7M5SKYFQSVUHWFWFQVC52SQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.680568836 (240h28m14.49980485s) [noise=true , address=185.167.204.205:4033 , piece_id=BVOQ7VXKB65CRU3M3APCYJ3NKFG6HG6UGNNU7UKI4CJYCYL5HY2Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.681606178 (240h28m14.500842202s) [noise=true , address=185.167.204.205:4004 , piece_id=UATYU6M64CYLYFO7ZGZZTMY6TUJM34WGYM5U4FKE2EADZ3WFHHXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.682849166 (240h28m14.50208519s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.683007592 (240h28m14.502243616s) [noise=true , address=185.167.204.205:4003 , piece_id=OAJS25SOIZDYBJK43WRYDDMO3OYR6P6NCE7O37KN7X2PICJNOZXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.684400130 (240h28m14.503636154s) [noise=true , address=185.167.204.205:4036 , piece_id=RMXRKSJM5DRMODM2CAZBXTKUMFTG4B4O26TMKC233WIOTRSR6KRQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.686095452 (240h28m14.505331466s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.747734833 (240h28m14.566970857s) [noise=true , address=81.171.14.70:4046 , piece_id=4NMIQBBHLZX3AZRNTOHSJ2LM5A26OV66FL7HENHJTGJPFZPDZO4A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:28.990034383 (240h28m15.809270407s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.104032302 (240h28m15.923268326s) [noise=true , address=89.149.201.147:4011 , piece_id=VJZIZ4X7SKEQCXU665VXZLLQ4IF5I2S5CAY45IZSIUG5X4JKPZFA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.227923806 (240h28m16.04715983s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.327415309 (240h28m16.146651333s) [noise=true , address=81.171.14.70:4044 , piece_id=I2ML43UPUBAZWL6LNXEXOXVCO4VMHQULA6LIZ6QWSDES4DC4RMYA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:30.946412145 (240h28m17.765648159s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.310420672 (240h28m18.129656686s) [noise=true , address=89.149.201.147:4018 , piece_id=QQ4VAJ2AKLWWRV4FSPUMYB4CW64GZNVDBCHHF2FLLPUFKFOELHMA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.578914230 (240h28m18.398150274s) [noise=true , address=81.171.14.70:4030 , piece_id=RGRX2LD7VJUKZUI7OTHL54N4TCX6BT27SG2656CHNK3C4USIM3XA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:32.696287178 (240h28m19.515523202s) [noise=true , address=89.149.201.147:4004 , piece_id=YYIMUGDQSCZBIYNQRDQHY3FCHGDUQ7NYMD2P77N2KJTMHVCC5NLA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.122355826 (240h28m19.94159185s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622613706 (240h28m20.44184973s) [noise=true , address=81.171.14.70:4045 , piece_id=OXF6AHOJRIBSLIM2AHDMWHJELCXJJOK4FDUCZX4R2IWKSFIT4GTQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622631460 (240h28m20.441867484s) [seg_pos={0 0}, upload=230887]: Satisfied cohort requirements after 26 pieces
Apr 17 16:14:33.622880831 (240h28m20.442116865s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC): protocol: read tcp 188.166.225.170:33320->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622889879 (240h28m20.442125913s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622919847 (240h28m20.442155871s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jcAHt22bETT27CCXJ8ZaaYxxoKBonBYsFcoq6A34DVVjvhQAN): protocol: read tcp 188.166.225.170:44412->81.171.14.70:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622924756 (240h28m20.44216078s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622936359 (240h28m20.442172383s) [seg_pos={0 0}, upload=230887]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 17 16:14:33.622990012 (240h28m20.442226036s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 17 16:14:33.622993769 (240h28m20.442229793s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 17 16:14:33.857773622 (240h28m20.677009646s) [upload=230887]: Done uploading segments...
Apr 17 16:14:33.857791256 (240h28m20.67702728s) [upload=230887]: Upload finished. err: <nil>
Duration: 4.139876919s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
4.462746768s trace:
https://jaeger.dev.storj.io/trace/7f8a8e8c290cfce4
FAIL (2026-04-17 16:55:45 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: context deadline exceeded
======LOGS======
}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468778967 (240h28m13.288014991s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468982109 (240h28m13.288218133s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469034200 (240h28m13.288270224s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469278090 (240h28m13.288514114s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469317958 (240h28m13.288553982s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469533453 (240h28m13.288769477s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469783256 (240h28m13.28901928s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470149613 (240h28m13.289385637s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470407480 (240h28m13.289643504s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:27.287464596 (240h28m14.10670062s) [upload=230887]: Next returned nil segment
Apr 17 16:14:27.287476399 (240h28m14.106712423s) [upload=230887]: Waiting for error group managing segments...
Apr 17 16:14:27.667935875 (240h28m14.487171899s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.669691854 (240h28m14.488927878s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.672696102 (240h28m14.491932116s) [noise=true , address=185.167.204.206:4044 , piece_id=PV5G7HUDQCIZRBRC4U4QBSXDPYMS6OKELUO6T376WSCHIJEKSSHA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.673873083 (240h28m14.493109108s) [noise=true , address=185.167.204.205:4014 , piece_id=NZ2QB3PRHEREH7L3JN6QWWKEUSCCZJPX4KX5TOBBTEFI7V77CLDQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674467470 (240h28m14.493703494s) [noise=true , address=185.167.204.206:4041 , piece_id=5KYRFUXZKMCXVC6I2XF3MWFFFCBKRDEQU32J46PELU2K7ITHDJCQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674861982 (240h28m14.494098006s) [noise=true , address=185.167.204.205:4023 , piece_id=GX2SDFS6H5G3OV5K3JPMX4AFDJJWQAZZMJWCFCDLIQJ6J2EEGBRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675219832 (240h28m14.494455856s) [noise=true , address=185.167.204.206:4062 , piece_id=JLJJYF2HITUHLDWANNO7TANPR2B55XBEPI73RGIEQWQAUXVAUOLQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675769814 (240h28m14.495005838s) [noise=true , address=185.167.204.205:4034 , piece_id=2YLRAMZ4EUMTBIAOP47JESS2CNUE72WD2X7WNS3G5SNXTSIKZE6Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.679234931 (240h28m14.498470955s) [noise=true , address=185.167.204.205:4005 , piece_id=K5BMMNJJLPMJJ37OERBC7A2Y2GS6O7M5SKYFQSVUHWFWFQVC52SQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.680568836 (240h28m14.49980485s) [noise=true , address=185.167.204.205:4033 , piece_id=BVOQ7VXKB65CRU3M3APCYJ3NKFG6HG6UGNNU7UKI4CJYCYL5HY2Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.681606178 (240h28m14.500842202s) [noise=true , address=185.167.204.205:4004 , piece_id=UATYU6M64CYLYFO7ZGZZTMY6TUJM34WGYM5U4FKE2EADZ3WFHHXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.682849166 (240h28m14.50208519s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.683007592 (240h28m14.502243616s) [noise=true , address=185.167.204.205:4003 , piece_id=OAJS25SOIZDYBJK43WRYDDMO3OYR6P6NCE7O37KN7X2PICJNOZXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.684400130 (240h28m14.503636154s) [noise=true , address=185.167.204.205:4036 , piece_id=RMXRKSJM5DRMODM2CAZBXTKUMFTG4B4O26TMKC233WIOTRSR6KRQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.686095452 (240h28m14.505331466s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.747734833 (240h28m14.566970857s) [noise=true , address=81.171.14.70:4046 , piece_id=4NMIQBBHLZX3AZRNTOHSJ2LM5A26OV66FL7HENHJTGJPFZPDZO4A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:28.990034383 (240h28m15.809270407s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.104032302 (240h28m15.923268326s) [noise=true , address=89.149.201.147:4011 , piece_id=VJZIZ4X7SKEQCXU665VXZLLQ4IF5I2S5CAY45IZSIUG5X4JKPZFA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.227923806 (240h28m16.04715983s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.327415309 (240h28m16.146651333s) [noise=true , address=81.171.14.70:4044 , piece_id=I2ML43UPUBAZWL6LNXEXOXVCO4VMHQULA6LIZ6QWSDES4DC4RMYA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:30.946412145 (240h28m17.765648159s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.310420672 (240h28m18.129656686s) [noise=true , address=89.149.201.147:4018 , piece_id=QQ4VAJ2AKLWWRV4FSPUMYB4CW64GZNVDBCHHF2FLLPUFKFOELHMA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.578914230 (240h28m18.398150274s) [noise=true , address=81.171.14.70:4030 , piece_id=RGRX2LD7VJUKZUI7OTHL54N4TCX6BT27SG2656CHNK3C4USIM3XA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:32.696287178 (240h28m19.515523202s) [noise=true , address=89.149.201.147:4004 , piece_id=YYIMUGDQSCZBIYNQRDQHY3FCHGDUQ7NYMD2P77N2KJTMHVCC5NLA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.122355826 (240h28m19.94159185s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622613706 (240h28m20.44184973s) [noise=true , address=81.171.14.70:4045 , piece_id=OXF6AHOJRIBSLIM2AHDMWHJELCXJJOK4FDUCZX4R2IWKSFIT4GTQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622631460 (240h28m20.441867484s) [seg_pos={0 0}, upload=230887]: Satisfied cohort requirements after 26 pieces
Apr 17 16:14:33.622880831 (240h28m20.442116865s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC): protocol: read tcp 188.166.225.170:33320->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622889879 (240h28m20.442125913s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622919847 (240h28m20.442155871s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jcAHt22bETT27CCXJ8ZaaYxxoKBonBYsFcoq6A34DVVjvhQAN): protocol: read tcp 188.166.225.170:44412->81.171.14.70:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622924756 (240h28m20.44216078s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622936359 (240h28m20.442172383s) [seg_pos={0 0}, upload=230887]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 17 16:14:33.622990012 (240h28m20.442226036s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 17 16:14:33.622993769 (240h28m20.442229793s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 17 16:14:33.857773622 (240h28m20.677009646s) [upload=230887]: Done uploading segments...
Apr 17 16:14:33.857791256 (240h28m20.67702728s) [upload=230887]: Upload finished. err: <nil>
Duration: 4.462777287s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
5.930743082s trace:
https://jaeger.dev.storj.io/trace/1c80c1acf9bc1e4e
FAIL (2026-04-17 16:55:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.93077275s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
4.260283839s trace:
https://jaeger.dev.storj.io/trace/7f8a8e8c290cfce4
FAIL (2026-04-17 16:55:16 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: context deadline exceeded
======LOGS======
}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468778967 (240h28m13.288014991s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468982109 (240h28m13.288218133s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469034200 (240h28m13.288270224s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469278090 (240h28m13.288514114s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469317958 (240h28m13.288553982s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469533453 (240h28m13.288769477s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469783256 (240h28m13.28901928s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470149613 (240h28m13.289385637s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470407480 (240h28m13.289643504s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:27.287464596 (240h28m14.10670062s) [upload=230887]: Next returned nil segment
Apr 17 16:14:27.287476399 (240h28m14.106712423s) [upload=230887]: Waiting for error group managing segments...
Apr 17 16:14:27.667935875 (240h28m14.487171899s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.669691854 (240h28m14.488927878s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.672696102 (240h28m14.491932116s) [noise=true , address=185.167.204.206:4044 , piece_id=PV5G7HUDQCIZRBRC4U4QBSXDPYMS6OKELUO6T376WSCHIJEKSSHA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.673873083 (240h28m14.493109108s) [noise=true , address=185.167.204.205:4014 , piece_id=NZ2QB3PRHEREH7L3JN6QWWKEUSCCZJPX4KX5TOBBTEFI7V77CLDQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674467470 (240h28m14.493703494s) [noise=true , address=185.167.204.206:4041 , piece_id=5KYRFUXZKMCXVC6I2XF3MWFFFCBKRDEQU32J46PELU2K7ITHDJCQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674861982 (240h28m14.494098006s) [noise=true , address=185.167.204.205:4023 , piece_id=GX2SDFS6H5G3OV5K3JPMX4AFDJJWQAZZMJWCFCDLIQJ6J2EEGBRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675219832 (240h28m14.494455856s) [noise=true , address=185.167.204.206:4062 , piece_id=JLJJYF2HITUHLDWANNO7TANPR2B55XBEPI73RGIEQWQAUXVAUOLQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675769814 (240h28m14.495005838s) [noise=true , address=185.167.204.205:4034 , piece_id=2YLRAMZ4EUMTBIAOP47JESS2CNUE72WD2X7WNS3G5SNXTSIKZE6Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.679234931 (240h28m14.498470955s) [noise=true , address=185.167.204.205:4005 , piece_id=K5BMMNJJLPMJJ37OERBC7A2Y2GS6O7M5SKYFQSVUHWFWFQVC52SQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.680568836 (240h28m14.49980485s) [noise=true , address=185.167.204.205:4033 , piece_id=BVOQ7VXKB65CRU3M3APCYJ3NKFG6HG6UGNNU7UKI4CJYCYL5HY2Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.681606178 (240h28m14.500842202s) [noise=true , address=185.167.204.205:4004 , piece_id=UATYU6M64CYLYFO7ZGZZTMY6TUJM34WGYM5U4FKE2EADZ3WFHHXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.682849166 (240h28m14.50208519s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.683007592 (240h28m14.502243616s) [noise=true , address=185.167.204.205:4003 , piece_id=OAJS25SOIZDYBJK43WRYDDMO3OYR6P6NCE7O37KN7X2PICJNOZXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.684400130 (240h28m14.503636154s) [noise=true , address=185.167.204.205:4036 , piece_id=RMXRKSJM5DRMODM2CAZBXTKUMFTG4B4O26TMKC233WIOTRSR6KRQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.686095452 (240h28m14.505331466s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.747734833 (240h28m14.566970857s) [noise=true , address=81.171.14.70:4046 , piece_id=4NMIQBBHLZX3AZRNTOHSJ2LM5A26OV66FL7HENHJTGJPFZPDZO4A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:28.990034383 (240h28m15.809270407s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.104032302 (240h28m15.923268326s) [noise=true , address=89.149.201.147:4011 , piece_id=VJZIZ4X7SKEQCXU665VXZLLQ4IF5I2S5CAY45IZSIUG5X4JKPZFA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.227923806 (240h28m16.04715983s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.327415309 (240h28m16.146651333s) [noise=true , address=81.171.14.70:4044 , piece_id=I2ML43UPUBAZWL6LNXEXOXVCO4VMHQULA6LIZ6QWSDES4DC4RMYA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:30.946412145 (240h28m17.765648159s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.310420672 (240h28m18.129656686s) [noise=true , address=89.149.201.147:4018 , piece_id=QQ4VAJ2AKLWWRV4FSPUMYB4CW64GZNVDBCHHF2FLLPUFKFOELHMA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.578914230 (240h28m18.398150274s) [noise=true , address=81.171.14.70:4030 , piece_id=RGRX2LD7VJUKZUI7OTHL54N4TCX6BT27SG2656CHNK3C4USIM3XA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:32.696287178 (240h28m19.515523202s) [noise=true , address=89.149.201.147:4004 , piece_id=YYIMUGDQSCZBIYNQRDQHY3FCHGDUQ7NYMD2P77N2KJTMHVCC5NLA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.122355826 (240h28m19.94159185s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622613706 (240h28m20.44184973s) [noise=true , address=81.171.14.70:4045 , piece_id=OXF6AHOJRIBSLIM2AHDMWHJELCXJJOK4FDUCZX4R2IWKSFIT4GTQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622631460 (240h28m20.441867484s) [seg_pos={0 0}, upload=230887]: Satisfied cohort requirements after 26 pieces
Apr 17 16:14:33.622880831 (240h28m20.442116865s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC): protocol: read tcp 188.166.225.170:33320->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622889879 (240h28m20.442125913s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622919847 (240h28m20.442155871s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jcAHt22bETT27CCXJ8ZaaYxxoKBonBYsFcoq6A34DVVjvhQAN): protocol: read tcp 188.166.225.170:44412->81.171.14.70:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622924756 (240h28m20.44216078s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622936359 (240h28m20.442172383s) [seg_pos={0 0}, upload=230887]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 17 16:14:33.622990012 (240h28m20.442226036s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 17 16:14:33.622993769 (240h28m20.442229793s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 17 16:14:33.857773622 (240h28m20.677009646s) [upload=230887]: Done uploading segments...
Apr 17 16:14:33.857791256 (240h28m20.67702728s) [upload=230887]: Upload finished. err: <nil>
Duration: 4.260307224s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
5.869393144s trace:
https://jaeger.dev.storj.io/trace/1c80c1acf9bc1e4e
FAIL (2026-04-17 16:55:00 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.869423994s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
5.721061092s trace:
https://jaeger.dev.storj.io/trace/1c80c1acf9bc1e4e
FAIL (2026-04-17 16:54:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.721088545s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
4.537041644s trace:
https://jaeger.dev.storj.io/trace/7f8a8e8c290cfce4
FAIL (2026-04-17 16:54:07 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: context deadline exceeded
======LOGS======
}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468778967 (240h28m13.288014991s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468982109 (240h28m13.288218133s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469034200 (240h28m13.288270224s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469278090 (240h28m13.288514114s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469317958 (240h28m13.288553982s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469533453 (240h28m13.288769477s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469783256 (240h28m13.28901928s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470149613 (240h28m13.289385637s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470407480 (240h28m13.289643504s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:27.287464596 (240h28m14.10670062s) [upload=230887]: Next returned nil segment
Apr 17 16:14:27.287476399 (240h28m14.106712423s) [upload=230887]: Waiting for error group managing segments...
Apr 17 16:14:27.667935875 (240h28m14.487171899s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.669691854 (240h28m14.488927878s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.672696102 (240h28m14.491932116s) [noise=true , address=185.167.204.206:4044 , piece_id=PV5G7HUDQCIZRBRC4U4QBSXDPYMS6OKELUO6T376WSCHIJEKSSHA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.673873083 (240h28m14.493109108s) [noise=true , address=185.167.204.205:4014 , piece_id=NZ2QB3PRHEREH7L3JN6QWWKEUSCCZJPX4KX5TOBBTEFI7V77CLDQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674467470 (240h28m14.493703494s) [noise=true , address=185.167.204.206:4041 , piece_id=5KYRFUXZKMCXVC6I2XF3MWFFFCBKRDEQU32J46PELU2K7ITHDJCQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674861982 (240h28m14.494098006s) [noise=true , address=185.167.204.205:4023 , piece_id=GX2SDFS6H5G3OV5K3JPMX4AFDJJWQAZZMJWCFCDLIQJ6J2EEGBRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675219832 (240h28m14.494455856s) [noise=true , address=185.167.204.206:4062 , piece_id=JLJJYF2HITUHLDWANNO7TANPR2B55XBEPI73RGIEQWQAUXVAUOLQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675769814 (240h28m14.495005838s) [noise=true , address=185.167.204.205:4034 , piece_id=2YLRAMZ4EUMTBIAOP47JESS2CNUE72WD2X7WNS3G5SNXTSIKZE6Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.679234931 (240h28m14.498470955s) [noise=true , address=185.167.204.205:4005 , piece_id=K5BMMNJJLPMJJ37OERBC7A2Y2GS6O7M5SKYFQSVUHWFWFQVC52SQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.680568836 (240h28m14.49980485s) [noise=true , address=185.167.204.205:4033 , piece_id=BVOQ7VXKB65CRU3M3APCYJ3NKFG6HG6UGNNU7UKI4CJYCYL5HY2Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.681606178 (240h28m14.500842202s) [noise=true , address=185.167.204.205:4004 , piece_id=UATYU6M64CYLYFO7ZGZZTMY6TUJM34WGYM5U4FKE2EADZ3WFHHXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.682849166 (240h28m14.50208519s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.683007592 (240h28m14.502243616s) [noise=true , address=185.167.204.205:4003 , piece_id=OAJS25SOIZDYBJK43WRYDDMO3OYR6P6NCE7O37KN7X2PICJNOZXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.684400130 (240h28m14.503636154s) [noise=true , address=185.167.204.205:4036 , piece_id=RMXRKSJM5DRMODM2CAZBXTKUMFTG4B4O26TMKC233WIOTRSR6KRQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.686095452 (240h28m14.505331466s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.747734833 (240h28m14.566970857s) [noise=true , address=81.171.14.70:4046 , piece_id=4NMIQBBHLZX3AZRNTOHSJ2LM5A26OV66FL7HENHJTGJPFZPDZO4A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:28.990034383 (240h28m15.809270407s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.104032302 (240h28m15.923268326s) [noise=true , address=89.149.201.147:4011 , piece_id=VJZIZ4X7SKEQCXU665VXZLLQ4IF5I2S5CAY45IZSIUG5X4JKPZFA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.227923806 (240h28m16.04715983s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.327415309 (240h28m16.146651333s) [noise=true , address=81.171.14.70:4044 , piece_id=I2ML43UPUBAZWL6LNXEXOXVCO4VMHQULA6LIZ6QWSDES4DC4RMYA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:30.946412145 (240h28m17.765648159s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.310420672 (240h28m18.129656686s) [noise=true , address=89.149.201.147:4018 , piece_id=QQ4VAJ2AKLWWRV4FSPUMYB4CW64GZNVDBCHHF2FLLPUFKFOELHMA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.578914230 (240h28m18.398150274s) [noise=true , address=81.171.14.70:4030 , piece_id=RGRX2LD7VJUKZUI7OTHL54N4TCX6BT27SG2656CHNK3C4USIM3XA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:32.696287178 (240h28m19.515523202s) [noise=true , address=89.149.201.147:4004 , piece_id=YYIMUGDQSCZBIYNQRDQHY3FCHGDUQ7NYMD2P77N2KJTMHVCC5NLA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.122355826 (240h28m19.94159185s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622613706 (240h28m20.44184973s) [noise=true , address=81.171.14.70:4045 , piece_id=OXF6AHOJRIBSLIM2AHDMWHJELCXJJOK4FDUCZX4R2IWKSFIT4GTQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622631460 (240h28m20.441867484s) [seg_pos={0 0}, upload=230887]: Satisfied cohort requirements after 26 pieces
Apr 17 16:14:33.622880831 (240h28m20.442116865s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC): protocol: read tcp 188.166.225.170:33320->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622889879 (240h28m20.442125913s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622919847 (240h28m20.442155871s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jcAHt22bETT27CCXJ8ZaaYxxoKBonBYsFcoq6A34DVVjvhQAN): protocol: read tcp 188.166.225.170:44412->81.171.14.70:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622924756 (240h28m20.44216078s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622936359 (240h28m20.442172383s) [seg_pos={0 0}, upload=230887]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 17 16:14:33.622990012 (240h28m20.442226036s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 17 16:14:33.622993769 (240h28m20.442229793s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 17 16:14:33.857773622 (240h28m20.677009646s) [upload=230887]: Done uploading segments...
Apr 17 16:14:33.857791256 (240h28m20.67702728s) [upload=230887]: Upload finished. err: <nil>
Duration: 4.537068365s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
5.859386185s trace:
https://jaeger.dev.storj.io/trace/1c80c1acf9bc1e4e
FAIL (2026-04-17 16:53:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.859426043s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
4.234170665s trace:
https://jaeger.dev.storj.io/trace/7f8a8e8c290cfce4
FAIL (2026-04-17 16:53:37 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: context deadline exceeded
======LOGS======
}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468778967 (240h28m13.288014991s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468982109 (240h28m13.288218133s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469034200 (240h28m13.288270224s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469278090 (240h28m13.288514114s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469317958 (240h28m13.288553982s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469533453 (240h28m13.288769477s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469783256 (240h28m13.28901928s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470149613 (240h28m13.289385637s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470407480 (240h28m13.289643504s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:27.287464596 (240h28m14.10670062s) [upload=230887]: Next returned nil segment
Apr 17 16:14:27.287476399 (240h28m14.106712423s) [upload=230887]: Waiting for error group managing segments...
Apr 17 16:14:27.667935875 (240h28m14.487171899s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.669691854 (240h28m14.488927878s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.672696102 (240h28m14.491932116s) [noise=true , address=185.167.204.206:4044 , piece_id=PV5G7HUDQCIZRBRC4U4QBSXDPYMS6OKELUO6T376WSCHIJEKSSHA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.673873083 (240h28m14.493109108s) [noise=true , address=185.167.204.205:4014 , piece_id=NZ2QB3PRHEREH7L3JN6QWWKEUSCCZJPX4KX5TOBBTEFI7V77CLDQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674467470 (240h28m14.493703494s) [noise=true , address=185.167.204.206:4041 , piece_id=5KYRFUXZKMCXVC6I2XF3MWFFFCBKRDEQU32J46PELU2K7ITHDJCQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674861982 (240h28m14.494098006s) [noise=true , address=185.167.204.205:4023 , piece_id=GX2SDFS6H5G3OV5K3JPMX4AFDJJWQAZZMJWCFCDLIQJ6J2EEGBRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675219832 (240h28m14.494455856s) [noise=true , address=185.167.204.206:4062 , piece_id=JLJJYF2HITUHLDWANNO7TANPR2B55XBEPI73RGIEQWQAUXVAUOLQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675769814 (240h28m14.495005838s) [noise=true , address=185.167.204.205:4034 , piece_id=2YLRAMZ4EUMTBIAOP47JESS2CNUE72WD2X7WNS3G5SNXTSIKZE6Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.679234931 (240h28m14.498470955s) [noise=true , address=185.167.204.205:4005 , piece_id=K5BMMNJJLPMJJ37OERBC7A2Y2GS6O7M5SKYFQSVUHWFWFQVC52SQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.680568836 (240h28m14.49980485s) [noise=true , address=185.167.204.205:4033 , piece_id=BVOQ7VXKB65CRU3M3APCYJ3NKFG6HG6UGNNU7UKI4CJYCYL5HY2Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.681606178 (240h28m14.500842202s) [noise=true , address=185.167.204.205:4004 , piece_id=UATYU6M64CYLYFO7ZGZZTMY6TUJM34WGYM5U4FKE2EADZ3WFHHXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.682849166 (240h28m14.50208519s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.683007592 (240h28m14.502243616s) [noise=true , address=185.167.204.205:4003 , piece_id=OAJS25SOIZDYBJK43WRYDDMO3OYR6P6NCE7O37KN7X2PICJNOZXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.684400130 (240h28m14.503636154s) [noise=true , address=185.167.204.205:4036 , piece_id=RMXRKSJM5DRMODM2CAZBXTKUMFTG4B4O26TMKC233WIOTRSR6KRQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.686095452 (240h28m14.505331466s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.747734833 (240h28m14.566970857s) [noise=true , address=81.171.14.70:4046 , piece_id=4NMIQBBHLZX3AZRNTOHSJ2LM5A26OV66FL7HENHJTGJPFZPDZO4A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:28.990034383 (240h28m15.809270407s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.104032302 (240h28m15.923268326s) [noise=true , address=89.149.201.147:4011 , piece_id=VJZIZ4X7SKEQCXU665VXZLLQ4IF5I2S5CAY45IZSIUG5X4JKPZFA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.227923806 (240h28m16.04715983s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.327415309 (240h28m16.146651333s) [noise=true , address=81.171.14.70:4044 , piece_id=I2ML43UPUBAZWL6LNXEXOXVCO4VMHQULA6LIZ6QWSDES4DC4RMYA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:30.946412145 (240h28m17.765648159s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.310420672 (240h28m18.129656686s) [noise=true , address=89.149.201.147:4018 , piece_id=QQ4VAJ2AKLWWRV4FSPUMYB4CW64GZNVDBCHHF2FLLPUFKFOELHMA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.578914230 (240h28m18.398150274s) [noise=true , address=81.171.14.70:4030 , piece_id=RGRX2LD7VJUKZUI7OTHL54N4TCX6BT27SG2656CHNK3C4USIM3XA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:32.696287178 (240h28m19.515523202s) [noise=true , address=89.149.201.147:4004 , piece_id=YYIMUGDQSCZBIYNQRDQHY3FCHGDUQ7NYMD2P77N2KJTMHVCC5NLA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.122355826 (240h28m19.94159185s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622613706 (240h28m20.44184973s) [noise=true , address=81.171.14.70:4045 , piece_id=OXF6AHOJRIBSLIM2AHDMWHJELCXJJOK4FDUCZX4R2IWKSFIT4GTQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622631460 (240h28m20.441867484s) [seg_pos={0 0}, upload=230887]: Satisfied cohort requirements after 26 pieces
Apr 17 16:14:33.622880831 (240h28m20.442116865s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC): protocol: read tcp 188.166.225.170:33320->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622889879 (240h28m20.442125913s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622919847 (240h28m20.442155871s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jcAHt22bETT27CCXJ8ZaaYxxoKBonBYsFcoq6A34DVVjvhQAN): protocol: read tcp 188.166.225.170:44412->81.171.14.70:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622924756 (240h28m20.44216078s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622936359 (240h28m20.442172383s) [seg_pos={0 0}, upload=230887]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 17 16:14:33.622990012 (240h28m20.442226036s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 17 16:14:33.622993769 (240h28m20.442229793s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 17 16:14:33.857773622 (240h28m20.677009646s) [upload=230887]: Done uploading segments...
Apr 17 16:14:33.857791256 (240h28m20.67702728s) [upload=230887]: Upload finished. err: <nil>
Duration: 4.234186184s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
5.68415476s trace:
https://jaeger.dev.storj.io/trace/1c80c1acf9bc1e4e
FAIL (2026-04-17 16:53:05 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.684174178s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
4.569724384s trace:
https://jaeger.dev.storj.io/trace/7f8a8e8c290cfce4
FAIL (2026-04-17 16:52:38 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: context deadline exceeded
======LOGS======
}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468778967 (240h28m13.288014991s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468982109 (240h28m13.288218133s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469034200 (240h28m13.288270224s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469278090 (240h28m13.288514114s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469317958 (240h28m13.288553982s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469533453 (240h28m13.288769477s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469783256 (240h28m13.28901928s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470149613 (240h28m13.289385637s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470407480 (240h28m13.289643504s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:27.287464596 (240h28m14.10670062s) [upload=230887]: Next returned nil segment
Apr 17 16:14:27.287476399 (240h28m14.106712423s) [upload=230887]: Waiting for error group managing segments...
Apr 17 16:14:27.667935875 (240h28m14.487171899s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.669691854 (240h28m14.488927878s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.672696102 (240h28m14.491932116s) [noise=true , address=185.167.204.206:4044 , piece_id=PV5G7HUDQCIZRBRC4U4QBSXDPYMS6OKELUO6T376WSCHIJEKSSHA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.673873083 (240h28m14.493109108s) [noise=true , address=185.167.204.205:4014 , piece_id=NZ2QB3PRHEREH7L3JN6QWWKEUSCCZJPX4KX5TOBBTEFI7V77CLDQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674467470 (240h28m14.493703494s) [noise=true , address=185.167.204.206:4041 , piece_id=5KYRFUXZKMCXVC6I2XF3MWFFFCBKRDEQU32J46PELU2K7ITHDJCQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674861982 (240h28m14.494098006s) [noise=true , address=185.167.204.205:4023 , piece_id=GX2SDFS6H5G3OV5K3JPMX4AFDJJWQAZZMJWCFCDLIQJ6J2EEGBRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675219832 (240h28m14.494455856s) [noise=true , address=185.167.204.206:4062 , piece_id=JLJJYF2HITUHLDWANNO7TANPR2B55XBEPI73RGIEQWQAUXVAUOLQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675769814 (240h28m14.495005838s) [noise=true , address=185.167.204.205:4034 , piece_id=2YLRAMZ4EUMTBIAOP47JESS2CNUE72WD2X7WNS3G5SNXTSIKZE6Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.679234931 (240h28m14.498470955s) [noise=true , address=185.167.204.205:4005 , piece_id=K5BMMNJJLPMJJ37OERBC7A2Y2GS6O7M5SKYFQSVUHWFWFQVC52SQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.680568836 (240h28m14.49980485s) [noise=true , address=185.167.204.205:4033 , piece_id=BVOQ7VXKB65CRU3M3APCYJ3NKFG6HG6UGNNU7UKI4CJYCYL5HY2Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.681606178 (240h28m14.500842202s) [noise=true , address=185.167.204.205:4004 , piece_id=UATYU6M64CYLYFO7ZGZZTMY6TUJM34WGYM5U4FKE2EADZ3WFHHXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.682849166 (240h28m14.50208519s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.683007592 (240h28m14.502243616s) [noise=true , address=185.167.204.205:4003 , piece_id=OAJS25SOIZDYBJK43WRYDDMO3OYR6P6NCE7O37KN7X2PICJNOZXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.684400130 (240h28m14.503636154s) [noise=true , address=185.167.204.205:4036 , piece_id=RMXRKSJM5DRMODM2CAZBXTKUMFTG4B4O26TMKC233WIOTRSR6KRQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.686095452 (240h28m14.505331466s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.747734833 (240h28m14.566970857s) [noise=true , address=81.171.14.70:4046 , piece_id=4NMIQBBHLZX3AZRNTOHSJ2LM5A26OV66FL7HENHJTGJPFZPDZO4A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:28.990034383 (240h28m15.809270407s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.104032302 (240h28m15.923268326s) [noise=true , address=89.149.201.147:4011 , piece_id=VJZIZ4X7SKEQCXU665VXZLLQ4IF5I2S5CAY45IZSIUG5X4JKPZFA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.227923806 (240h28m16.04715983s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.327415309 (240h28m16.146651333s) [noise=true , address=81.171.14.70:4044 , piece_id=I2ML43UPUBAZWL6LNXEXOXVCO4VMHQULA6LIZ6QWSDES4DC4RMYA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:30.946412145 (240h28m17.765648159s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.310420672 (240h28m18.129656686s) [noise=true , address=89.149.201.147:4018 , piece_id=QQ4VAJ2AKLWWRV4FSPUMYB4CW64GZNVDBCHHF2FLLPUFKFOELHMA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.578914230 (240h28m18.398150274s) [noise=true , address=81.171.14.70:4030 , piece_id=RGRX2LD7VJUKZUI7OTHL54N4TCX6BT27SG2656CHNK3C4USIM3XA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:32.696287178 (240h28m19.515523202s) [noise=true , address=89.149.201.147:4004 , piece_id=YYIMUGDQSCZBIYNQRDQHY3FCHGDUQ7NYMD2P77N2KJTMHVCC5NLA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.122355826 (240h28m19.94159185s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622613706 (240h28m20.44184973s) [noise=true , address=81.171.14.70:4045 , piece_id=OXF6AHOJRIBSLIM2AHDMWHJELCXJJOK4FDUCZX4R2IWKSFIT4GTQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622631460 (240h28m20.441867484s) [seg_pos={0 0}, upload=230887]: Satisfied cohort requirements after 26 pieces
Apr 17 16:14:33.622880831 (240h28m20.442116865s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC): protocol: read tcp 188.166.225.170:33320->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622889879 (240h28m20.442125913s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622919847 (240h28m20.442155871s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jcAHt22bETT27CCXJ8ZaaYxxoKBonBYsFcoq6A34DVVjvhQAN): protocol: read tcp 188.166.225.170:44412->81.171.14.70:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622924756 (240h28m20.44216078s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622936359 (240h28m20.442172383s) [seg_pos={0 0}, upload=230887]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 17 16:14:33.622990012 (240h28m20.442226036s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 17 16:14:33.622993769 (240h28m20.442229793s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 17 16:14:33.857773622 (240h28m20.677009646s) [upload=230887]: Done uploading segments...
Apr 17 16:14:33.857791256 (240h28m20.67702728s) [upload=230887]: Upload finished. err: <nil>
Duration: 4.569751957s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
5.683272742s trace:
https://jaeger.dev.storj.io/trace/1c80c1acf9bc1e4e
FAIL (2026-04-17 16:52:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.683297811s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
5.729050502s trace:
https://jaeger.dev.storj.io/trace/1c80c1acf9bc1e4e
FAIL (2026-04-17 16:51:50 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.729073657s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
4.541528405s trace:
https://jaeger.dev.storj.io/trace/7f8a8e8c290cfce4
FAIL (2026-04-17 16:51:47 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: context deadline exceeded
======LOGS======
}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468778967 (240h28m13.288014991s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468982109 (240h28m13.288218133s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469034200 (240h28m13.288270224s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469278090 (240h28m13.288514114s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469317958 (240h28m13.288553982s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469533453 (240h28m13.288769477s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469783256 (240h28m13.28901928s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470149613 (240h28m13.289385637s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470407480 (240h28m13.289643504s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:27.287464596 (240h28m14.10670062s) [upload=230887]: Next returned nil segment
Apr 17 16:14:27.287476399 (240h28m14.106712423s) [upload=230887]: Waiting for error group managing segments...
Apr 17 16:14:27.667935875 (240h28m14.487171899s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.669691854 (240h28m14.488927878s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.672696102 (240h28m14.491932116s) [noise=true , address=185.167.204.206:4044 , piece_id=PV5G7HUDQCIZRBRC4U4QBSXDPYMS6OKELUO6T376WSCHIJEKSSHA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.673873083 (240h28m14.493109108s) [noise=true , address=185.167.204.205:4014 , piece_id=NZ2QB3PRHEREH7L3JN6QWWKEUSCCZJPX4KX5TOBBTEFI7V77CLDQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674467470 (240h28m14.493703494s) [noise=true , address=185.167.204.206:4041 , piece_id=5KYRFUXZKMCXVC6I2XF3MWFFFCBKRDEQU32J46PELU2K7ITHDJCQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674861982 (240h28m14.494098006s) [noise=true , address=185.167.204.205:4023 , piece_id=GX2SDFS6H5G3OV5K3JPMX4AFDJJWQAZZMJWCFCDLIQJ6J2EEGBRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675219832 (240h28m14.494455856s) [noise=true , address=185.167.204.206:4062 , piece_id=JLJJYF2HITUHLDWANNO7TANPR2B55XBEPI73RGIEQWQAUXVAUOLQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675769814 (240h28m14.495005838s) [noise=true , address=185.167.204.205:4034 , piece_id=2YLRAMZ4EUMTBIAOP47JESS2CNUE72WD2X7WNS3G5SNXTSIKZE6Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.679234931 (240h28m14.498470955s) [noise=true , address=185.167.204.205:4005 , piece_id=K5BMMNJJLPMJJ37OERBC7A2Y2GS6O7M5SKYFQSVUHWFWFQVC52SQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.680568836 (240h28m14.49980485s) [noise=true , address=185.167.204.205:4033 , piece_id=BVOQ7VXKB65CRU3M3APCYJ3NKFG6HG6UGNNU7UKI4CJYCYL5HY2Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.681606178 (240h28m14.500842202s) [noise=true , address=185.167.204.205:4004 , piece_id=UATYU6M64CYLYFO7ZGZZTMY6TUJM34WGYM5U4FKE2EADZ3WFHHXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.682849166 (240h28m14.50208519s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.683007592 (240h28m14.502243616s) [noise=true , address=185.167.204.205:4003 , piece_id=OAJS25SOIZDYBJK43WRYDDMO3OYR6P6NCE7O37KN7X2PICJNOZXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.684400130 (240h28m14.503636154s) [noise=true , address=185.167.204.205:4036 , piece_id=RMXRKSJM5DRMODM2CAZBXTKUMFTG4B4O26TMKC233WIOTRSR6KRQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.686095452 (240h28m14.505331466s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.747734833 (240h28m14.566970857s) [noise=true , address=81.171.14.70:4046 , piece_id=4NMIQBBHLZX3AZRNTOHSJ2LM5A26OV66FL7HENHJTGJPFZPDZO4A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:28.990034383 (240h28m15.809270407s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.104032302 (240h28m15.923268326s) [noise=true , address=89.149.201.147:4011 , piece_id=VJZIZ4X7SKEQCXU665VXZLLQ4IF5I2S5CAY45IZSIUG5X4JKPZFA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.227923806 (240h28m16.04715983s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.327415309 (240h28m16.146651333s) [noise=true , address=81.171.14.70:4044 , piece_id=I2ML43UPUBAZWL6LNXEXOXVCO4VMHQULA6LIZ6QWSDES4DC4RMYA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:30.946412145 (240h28m17.765648159s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.310420672 (240h28m18.129656686s) [noise=true , address=89.149.201.147:4018 , piece_id=QQ4VAJ2AKLWWRV4FSPUMYB4CW64GZNVDBCHHF2FLLPUFKFOELHMA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.578914230 (240h28m18.398150274s) [noise=true , address=81.171.14.70:4030 , piece_id=RGRX2LD7VJUKZUI7OTHL54N4TCX6BT27SG2656CHNK3C4USIM3XA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:32.696287178 (240h28m19.515523202s) [noise=true , address=89.149.201.147:4004 , piece_id=YYIMUGDQSCZBIYNQRDQHY3FCHGDUQ7NYMD2P77N2KJTMHVCC5NLA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.122355826 (240h28m19.94159185s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622613706 (240h28m20.44184973s) [noise=true , address=81.171.14.70:4045 , piece_id=OXF6AHOJRIBSLIM2AHDMWHJELCXJJOK4FDUCZX4R2IWKSFIT4GTQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622631460 (240h28m20.441867484s) [seg_pos={0 0}, upload=230887]: Satisfied cohort requirements after 26 pieces
Apr 17 16:14:33.622880831 (240h28m20.442116865s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC): protocol: read tcp 188.166.225.170:33320->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622889879 (240h28m20.442125913s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622919847 (240h28m20.442155871s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jcAHt22bETT27CCXJ8ZaaYxxoKBonBYsFcoq6A34DVVjvhQAN): protocol: read tcp 188.166.225.170:44412->81.171.14.70:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622924756 (240h28m20.44216078s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622936359 (240h28m20.442172383s) [seg_pos={0 0}, upload=230887]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 17 16:14:33.622990012 (240h28m20.442226036s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 17 16:14:33.622993769 (240h28m20.442229793s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 17 16:14:33.857773622 (240h28m20.677009646s) [upload=230887]: Done uploading segments...
Apr 17 16:14:33.857791256 (240h28m20.67702728s) [upload=230887]: Upload finished. err: <nil>
Duration: 4.541553996s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
5.694644289s trace:
https://jaeger.dev.storj.io/trace/1c80c1acf9bc1e4e
FAIL (2026-04-17 16:50:46 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.694679977s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
4.192414074s trace:
https://jaeger.dev.storj.io/trace/7f8a8e8c290cfce4
FAIL (2026-04-17 16:50:44 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: context deadline exceeded
======LOGS======
}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468778967 (240h28m13.288014991s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468982109 (240h28m13.288218133s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469034200 (240h28m13.288270224s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469278090 (240h28m13.288514114s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469317958 (240h28m13.288553982s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469533453 (240h28m13.288769477s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469783256 (240h28m13.28901928s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470149613 (240h28m13.289385637s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470407480 (240h28m13.289643504s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:27.287464596 (240h28m14.10670062s) [upload=230887]: Next returned nil segment
Apr 17 16:14:27.287476399 (240h28m14.106712423s) [upload=230887]: Waiting for error group managing segments...
Apr 17 16:14:27.667935875 (240h28m14.487171899s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.669691854 (240h28m14.488927878s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.672696102 (240h28m14.491932116s) [noise=true , address=185.167.204.206:4044 , piece_id=PV5G7HUDQCIZRBRC4U4QBSXDPYMS6OKELUO6T376WSCHIJEKSSHA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.673873083 (240h28m14.493109108s) [noise=true , address=185.167.204.205:4014 , piece_id=NZ2QB3PRHEREH7L3JN6QWWKEUSCCZJPX4KX5TOBBTEFI7V77CLDQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674467470 (240h28m14.493703494s) [noise=true , address=185.167.204.206:4041 , piece_id=5KYRFUXZKMCXVC6I2XF3MWFFFCBKRDEQU32J46PELU2K7ITHDJCQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674861982 (240h28m14.494098006s) [noise=true , address=185.167.204.205:4023 , piece_id=GX2SDFS6H5G3OV5K3JPMX4AFDJJWQAZZMJWCFCDLIQJ6J2EEGBRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675219832 (240h28m14.494455856s) [noise=true , address=185.167.204.206:4062 , piece_id=JLJJYF2HITUHLDWANNO7TANPR2B55XBEPI73RGIEQWQAUXVAUOLQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675769814 (240h28m14.495005838s) [noise=true , address=185.167.204.205:4034 , piece_id=2YLRAMZ4EUMTBIAOP47JESS2CNUE72WD2X7WNS3G5SNXTSIKZE6Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.679234931 (240h28m14.498470955s) [noise=true , address=185.167.204.205:4005 , piece_id=K5BMMNJJLPMJJ37OERBC7A2Y2GS6O7M5SKYFQSVUHWFWFQVC52SQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.680568836 (240h28m14.49980485s) [noise=true , address=185.167.204.205:4033 , piece_id=BVOQ7VXKB65CRU3M3APCYJ3NKFG6HG6UGNNU7UKI4CJYCYL5HY2Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.681606178 (240h28m14.500842202s) [noise=true , address=185.167.204.205:4004 , piece_id=UATYU6M64CYLYFO7ZGZZTMY6TUJM34WGYM5U4FKE2EADZ3WFHHXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.682849166 (240h28m14.50208519s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.683007592 (240h28m14.502243616s) [noise=true , address=185.167.204.205:4003 , piece_id=OAJS25SOIZDYBJK43WRYDDMO3OYR6P6NCE7O37KN7X2PICJNOZXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.684400130 (240h28m14.503636154s) [noise=true , address=185.167.204.205:4036 , piece_id=RMXRKSJM5DRMODM2CAZBXTKUMFTG4B4O26TMKC233WIOTRSR6KRQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.686095452 (240h28m14.505331466s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.747734833 (240h28m14.566970857s) [noise=true , address=81.171.14.70:4046 , piece_id=4NMIQBBHLZX3AZRNTOHSJ2LM5A26OV66FL7HENHJTGJPFZPDZO4A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:28.990034383 (240h28m15.809270407s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.104032302 (240h28m15.923268326s) [noise=true , address=89.149.201.147:4011 , piece_id=VJZIZ4X7SKEQCXU665VXZLLQ4IF5I2S5CAY45IZSIUG5X4JKPZFA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.227923806 (240h28m16.04715983s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.327415309 (240h28m16.146651333s) [noise=true , address=81.171.14.70:4044 , piece_id=I2ML43UPUBAZWL6LNXEXOXVCO4VMHQULA6LIZ6QWSDES4DC4RMYA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:30.946412145 (240h28m17.765648159s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.310420672 (240h28m18.129656686s) [noise=true , address=89.149.201.147:4018 , piece_id=QQ4VAJ2AKLWWRV4FSPUMYB4CW64GZNVDBCHHF2FLLPUFKFOELHMA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.578914230 (240h28m18.398150274s) [noise=true , address=81.171.14.70:4030 , piece_id=RGRX2LD7VJUKZUI7OTHL54N4TCX6BT27SG2656CHNK3C4USIM3XA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:32.696287178 (240h28m19.515523202s) [noise=true , address=89.149.201.147:4004 , piece_id=YYIMUGDQSCZBIYNQRDQHY3FCHGDUQ7NYMD2P77N2KJTMHVCC5NLA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.122355826 (240h28m19.94159185s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622613706 (240h28m20.44184973s) [noise=true , address=81.171.14.70:4045 , piece_id=OXF6AHOJRIBSLIM2AHDMWHJELCXJJOK4FDUCZX4R2IWKSFIT4GTQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622631460 (240h28m20.441867484s) [seg_pos={0 0}, upload=230887]: Satisfied cohort requirements after 26 pieces
Apr 17 16:14:33.622880831 (240h28m20.442116865s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC): protocol: read tcp 188.166.225.170:33320->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622889879 (240h28m20.442125913s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622919847 (240h28m20.442155871s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jcAHt22bETT27CCXJ8ZaaYxxoKBonBYsFcoq6A34DVVjvhQAN): protocol: read tcp 188.166.225.170:44412->81.171.14.70:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622924756 (240h28m20.44216078s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622936359 (240h28m20.442172383s) [seg_pos={0 0}, upload=230887]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 17 16:14:33.622990012 (240h28m20.442226036s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 17 16:14:33.622993769 (240h28m20.442229793s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 17 16:14:33.857773622 (240h28m20.677009646s) [upload=230887]: Done uploading segments...
Apr 17 16:14:33.857791256 (240h28m20.67702728s) [upload=230887]: Upload finished. err: <nil>
Duration: 4.192441007s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
4.504080891s trace:
https://jaeger.dev.storj.io/trace/7f8a8e8c290cfce4
FAIL (2026-04-17 16:50:29 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: context deadline exceeded
======LOGS======
}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468778967 (240h28m13.288014991s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468982109 (240h28m13.288218133s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469034200 (240h28m13.288270224s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469278090 (240h28m13.288514114s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469317958 (240h28m13.288553982s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469533453 (240h28m13.288769477s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469783256 (240h28m13.28901928s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470149613 (240h28m13.289385637s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470407480 (240h28m13.289643504s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:27.287464596 (240h28m14.10670062s) [upload=230887]: Next returned nil segment
Apr 17 16:14:27.287476399 (240h28m14.106712423s) [upload=230887]: Waiting for error group managing segments...
Apr 17 16:14:27.667935875 (240h28m14.487171899s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.669691854 (240h28m14.488927878s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.672696102 (240h28m14.491932116s) [noise=true , address=185.167.204.206:4044 , piece_id=PV5G7HUDQCIZRBRC4U4QBSXDPYMS6OKELUO6T376WSCHIJEKSSHA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.673873083 (240h28m14.493109108s) [noise=true , address=185.167.204.205:4014 , piece_id=NZ2QB3PRHEREH7L3JN6QWWKEUSCCZJPX4KX5TOBBTEFI7V77CLDQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674467470 (240h28m14.493703494s) [noise=true , address=185.167.204.206:4041 , piece_id=5KYRFUXZKMCXVC6I2XF3MWFFFCBKRDEQU32J46PELU2K7ITHDJCQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674861982 (240h28m14.494098006s) [noise=true , address=185.167.204.205:4023 , piece_id=GX2SDFS6H5G3OV5K3JPMX4AFDJJWQAZZMJWCFCDLIQJ6J2EEGBRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675219832 (240h28m14.494455856s) [noise=true , address=185.167.204.206:4062 , piece_id=JLJJYF2HITUHLDWANNO7TANPR2B55XBEPI73RGIEQWQAUXVAUOLQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675769814 (240h28m14.495005838s) [noise=true , address=185.167.204.205:4034 , piece_id=2YLRAMZ4EUMTBIAOP47JESS2CNUE72WD2X7WNS3G5SNXTSIKZE6Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.679234931 (240h28m14.498470955s) [noise=true , address=185.167.204.205:4005 , piece_id=K5BMMNJJLPMJJ37OERBC7A2Y2GS6O7M5SKYFQSVUHWFWFQVC52SQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.680568836 (240h28m14.49980485s) [noise=true , address=185.167.204.205:4033 , piece_id=BVOQ7VXKB65CRU3M3APCYJ3NKFG6HG6UGNNU7UKI4CJYCYL5HY2Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.681606178 (240h28m14.500842202s) [noise=true , address=185.167.204.205:4004 , piece_id=UATYU6M64CYLYFO7ZGZZTMY6TUJM34WGYM5U4FKE2EADZ3WFHHXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.682849166 (240h28m14.50208519s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.683007592 (240h28m14.502243616s) [noise=true , address=185.167.204.205:4003 , piece_id=OAJS25SOIZDYBJK43WRYDDMO3OYR6P6NCE7O37KN7X2PICJNOZXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.684400130 (240h28m14.503636154s) [noise=true , address=185.167.204.205:4036 , piece_id=RMXRKSJM5DRMODM2CAZBXTKUMFTG4B4O26TMKC233WIOTRSR6KRQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.686095452 (240h28m14.505331466s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.747734833 (240h28m14.566970857s) [noise=true , address=81.171.14.70:4046 , piece_id=4NMIQBBHLZX3AZRNTOHSJ2LM5A26OV66FL7HENHJTGJPFZPDZO4A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:28.990034383 (240h28m15.809270407s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.104032302 (240h28m15.923268326s) [noise=true , address=89.149.201.147:4011 , piece_id=VJZIZ4X7SKEQCXU665VXZLLQ4IF5I2S5CAY45IZSIUG5X4JKPZFA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.227923806 (240h28m16.04715983s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.327415309 (240h28m16.146651333s) [noise=true , address=81.171.14.70:4044 , piece_id=I2ML43UPUBAZWL6LNXEXOXVCO4VMHQULA6LIZ6QWSDES4DC4RMYA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:30.946412145 (240h28m17.765648159s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.310420672 (240h28m18.129656686s) [noise=true , address=89.149.201.147:4018 , piece_id=QQ4VAJ2AKLWWRV4FSPUMYB4CW64GZNVDBCHHF2FLLPUFKFOELHMA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.578914230 (240h28m18.398150274s) [noise=true , address=81.171.14.70:4030 , piece_id=RGRX2LD7VJUKZUI7OTHL54N4TCX6BT27SG2656CHNK3C4USIM3XA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:32.696287178 (240h28m19.515523202s) [noise=true , address=89.149.201.147:4004 , piece_id=YYIMUGDQSCZBIYNQRDQHY3FCHGDUQ7NYMD2P77N2KJTMHVCC5NLA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.122355826 (240h28m19.94159185s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622613706 (240h28m20.44184973s) [noise=true , address=81.171.14.70:4045 , piece_id=OXF6AHOJRIBSLIM2AHDMWHJELCXJJOK4FDUCZX4R2IWKSFIT4GTQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622631460 (240h28m20.441867484s) [seg_pos={0 0}, upload=230887]: Satisfied cohort requirements after 26 pieces
Apr 17 16:14:33.622880831 (240h28m20.442116865s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC): protocol: read tcp 188.166.225.170:33320->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622889879 (240h28m20.442125913s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622919847 (240h28m20.442155871s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jcAHt22bETT27CCXJ8ZaaYxxoKBonBYsFcoq6A34DVVjvhQAN): protocol: read tcp 188.166.225.170:44412->81.171.14.70:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622924756 (240h28m20.44216078s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622936359 (240h28m20.442172383s) [seg_pos={0 0}, upload=230887]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 17 16:14:33.622990012 (240h28m20.442226036s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 17 16:14:33.622993769 (240h28m20.442229793s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 17 16:14:33.857773622 (240h28m20.677009646s) [upload=230887]: Done uploading segments...
Apr 17 16:14:33.857791256 (240h28m20.67702728s) [upload=230887]: Upload finished. err: <nil>
Duration: 4.504109356s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
6.086601468s trace:
https://jaeger.dev.storj.io/trace/1c80c1acf9bc1e4e
FAIL (2026-04-17 16:50:23 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.086632037s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
4.157848374s trace:
https://jaeger.dev.storj.io/trace/7f8a8e8c290cfce4
FAIL (2026-04-17 16:50:06 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: context deadline exceeded
======LOGS======
}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468778967 (240h28m13.288014991s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468982109 (240h28m13.288218133s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469034200 (240h28m13.288270224s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469278090 (240h28m13.288514114s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469317958 (240h28m13.288553982s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469533453 (240h28m13.288769477s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469783256 (240h28m13.28901928s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470149613 (240h28m13.289385637s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470407480 (240h28m13.289643504s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:27.287464596 (240h28m14.10670062s) [upload=230887]: Next returned nil segment
Apr 17 16:14:27.287476399 (240h28m14.106712423s) [upload=230887]: Waiting for error group managing segments...
Apr 17 16:14:27.667935875 (240h28m14.487171899s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.669691854 (240h28m14.488927878s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.672696102 (240h28m14.491932116s) [noise=true , address=185.167.204.206:4044 , piece_id=PV5G7HUDQCIZRBRC4U4QBSXDPYMS6OKELUO6T376WSCHIJEKSSHA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.673873083 (240h28m14.493109108s) [noise=true , address=185.167.204.205:4014 , piece_id=NZ2QB3PRHEREH7L3JN6QWWKEUSCCZJPX4KX5TOBBTEFI7V77CLDQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674467470 (240h28m14.493703494s) [noise=true , address=185.167.204.206:4041 , piece_id=5KYRFUXZKMCXVC6I2XF3MWFFFCBKRDEQU32J46PELU2K7ITHDJCQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674861982 (240h28m14.494098006s) [noise=true , address=185.167.204.205:4023 , piece_id=GX2SDFS6H5G3OV5K3JPMX4AFDJJWQAZZMJWCFCDLIQJ6J2EEGBRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675219832 (240h28m14.494455856s) [noise=true , address=185.167.204.206:4062 , piece_id=JLJJYF2HITUHLDWANNO7TANPR2B55XBEPI73RGIEQWQAUXVAUOLQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675769814 (240h28m14.495005838s) [noise=true , address=185.167.204.205:4034 , piece_id=2YLRAMZ4EUMTBIAOP47JESS2CNUE72WD2X7WNS3G5SNXTSIKZE6Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.679234931 (240h28m14.498470955s) [noise=true , address=185.167.204.205:4005 , piece_id=K5BMMNJJLPMJJ37OERBC7A2Y2GS6O7M5SKYFQSVUHWFWFQVC52SQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.680568836 (240h28m14.49980485s) [noise=true , address=185.167.204.205:4033 , piece_id=BVOQ7VXKB65CRU3M3APCYJ3NKFG6HG6UGNNU7UKI4CJYCYL5HY2Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.681606178 (240h28m14.500842202s) [noise=true , address=185.167.204.205:4004 , piece_id=UATYU6M64CYLYFO7ZGZZTMY6TUJM34WGYM5U4FKE2EADZ3WFHHXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.682849166 (240h28m14.50208519s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.683007592 (240h28m14.502243616s) [noise=true , address=185.167.204.205:4003 , piece_id=OAJS25SOIZDYBJK43WRYDDMO3OYR6P6NCE7O37KN7X2PICJNOZXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.684400130 (240h28m14.503636154s) [noise=true , address=185.167.204.205:4036 , piece_id=RMXRKSJM5DRMODM2CAZBXTKUMFTG4B4O26TMKC233WIOTRSR6KRQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.686095452 (240h28m14.505331466s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.747734833 (240h28m14.566970857s) [noise=true , address=81.171.14.70:4046 , piece_id=4NMIQBBHLZX3AZRNTOHSJ2LM5A26OV66FL7HENHJTGJPFZPDZO4A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:28.990034383 (240h28m15.809270407s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.104032302 (240h28m15.923268326s) [noise=true , address=89.149.201.147:4011 , piece_id=VJZIZ4X7SKEQCXU665VXZLLQ4IF5I2S5CAY45IZSIUG5X4JKPZFA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.227923806 (240h28m16.04715983s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.327415309 (240h28m16.146651333s) [noise=true , address=81.171.14.70:4044 , piece_id=I2ML43UPUBAZWL6LNXEXOXVCO4VMHQULA6LIZ6QWSDES4DC4RMYA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:30.946412145 (240h28m17.765648159s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.310420672 (240h28m18.129656686s) [noise=true , address=89.149.201.147:4018 , piece_id=QQ4VAJ2AKLWWRV4FSPUMYB4CW64GZNVDBCHHF2FLLPUFKFOELHMA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.578914230 (240h28m18.398150274s) [noise=true , address=81.171.14.70:4030 , piece_id=RGRX2LD7VJUKZUI7OTHL54N4TCX6BT27SG2656CHNK3C4USIM3XA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:32.696287178 (240h28m19.515523202s) [noise=true , address=89.149.201.147:4004 , piece_id=YYIMUGDQSCZBIYNQRDQHY3FCHGDUQ7NYMD2P77N2KJTMHVCC5NLA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.122355826 (240h28m19.94159185s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622613706 (240h28m20.44184973s) [noise=true , address=81.171.14.70:4045 , piece_id=OXF6AHOJRIBSLIM2AHDMWHJELCXJJOK4FDUCZX4R2IWKSFIT4GTQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622631460 (240h28m20.441867484s) [seg_pos={0 0}, upload=230887]: Satisfied cohort requirements after 26 pieces
Apr 17 16:14:33.622880831 (240h28m20.442116865s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC): protocol: read tcp 188.166.225.170:33320->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622889879 (240h28m20.442125913s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622919847 (240h28m20.442155871s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jcAHt22bETT27CCXJ8ZaaYxxoKBonBYsFcoq6A34DVVjvhQAN): protocol: read tcp 188.166.225.170:44412->81.171.14.70:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622924756 (240h28m20.44216078s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622936359 (240h28m20.442172383s) [seg_pos={0 0}, upload=230887]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 17 16:14:33.622990012 (240h28m20.442226036s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 17 16:14:33.622993769 (240h28m20.442229793s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 17 16:14:33.857773622 (240h28m20.677009646s) [upload=230887]: Done uploading segments...
Apr 17 16:14:33.857791256 (240h28m20.67702728s) [upload=230887]: Upload finished. err: <nil>
Duration: 4.15788216s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
3.657054905s trace:
https://jaeger.dev.storj.io/trace/7f8a8e8c290cfce4
FAIL (2026-04-17 16:49:33 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: context deadline exceeded
======LOGS======
}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468778967 (240h28m13.288014991s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468982109 (240h28m13.288218133s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469034200 (240h28m13.288270224s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469278090 (240h28m13.288514114s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469317958 (240h28m13.288553982s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469533453 (240h28m13.288769477s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469783256 (240h28m13.28901928s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470149613 (240h28m13.289385637s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470407480 (240h28m13.289643504s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:27.287464596 (240h28m14.10670062s) [upload=230887]: Next returned nil segment
Apr 17 16:14:27.287476399 (240h28m14.106712423s) [upload=230887]: Waiting for error group managing segments...
Apr 17 16:14:27.667935875 (240h28m14.487171899s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.669691854 (240h28m14.488927878s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.672696102 (240h28m14.491932116s) [noise=true , address=185.167.204.206:4044 , piece_id=PV5G7HUDQCIZRBRC4U4QBSXDPYMS6OKELUO6T376WSCHIJEKSSHA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.673873083 (240h28m14.493109108s) [noise=true , address=185.167.204.205:4014 , piece_id=NZ2QB3PRHEREH7L3JN6QWWKEUSCCZJPX4KX5TOBBTEFI7V77CLDQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674467470 (240h28m14.493703494s) [noise=true , address=185.167.204.206:4041 , piece_id=5KYRFUXZKMCXVC6I2XF3MWFFFCBKRDEQU32J46PELU2K7ITHDJCQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674861982 (240h28m14.494098006s) [noise=true , address=185.167.204.205:4023 , piece_id=GX2SDFS6H5G3OV5K3JPMX4AFDJJWQAZZMJWCFCDLIQJ6J2EEGBRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675219832 (240h28m14.494455856s) [noise=true , address=185.167.204.206:4062 , piece_id=JLJJYF2HITUHLDWANNO7TANPR2B55XBEPI73RGIEQWQAUXVAUOLQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675769814 (240h28m14.495005838s) [noise=true , address=185.167.204.205:4034 , piece_id=2YLRAMZ4EUMTBIAOP47JESS2CNUE72WD2X7WNS3G5SNXTSIKZE6Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.679234931 (240h28m14.498470955s) [noise=true , address=185.167.204.205:4005 , piece_id=K5BMMNJJLPMJJ37OERBC7A2Y2GS6O7M5SKYFQSVUHWFWFQVC52SQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.680568836 (240h28m14.49980485s) [noise=true , address=185.167.204.205:4033 , piece_id=BVOQ7VXKB65CRU3M3APCYJ3NKFG6HG6UGNNU7UKI4CJYCYL5HY2Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.681606178 (240h28m14.500842202s) [noise=true , address=185.167.204.205:4004 , piece_id=UATYU6M64CYLYFO7ZGZZTMY6TUJM34WGYM5U4FKE2EADZ3WFHHXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.682849166 (240h28m14.50208519s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.683007592 (240h28m14.502243616s) [noise=true , address=185.167.204.205:4003 , piece_id=OAJS25SOIZDYBJK43WRYDDMO3OYR6P6NCE7O37KN7X2PICJNOZXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.684400130 (240h28m14.503636154s) [noise=true , address=185.167.204.205:4036 , piece_id=RMXRKSJM5DRMODM2CAZBXTKUMFTG4B4O26TMKC233WIOTRSR6KRQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.686095452 (240h28m14.505331466s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.747734833 (240h28m14.566970857s) [noise=true , address=81.171.14.70:4046 , piece_id=4NMIQBBHLZX3AZRNTOHSJ2LM5A26OV66FL7HENHJTGJPFZPDZO4A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:28.990034383 (240h28m15.809270407s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.104032302 (240h28m15.923268326s) [noise=true , address=89.149.201.147:4011 , piece_id=VJZIZ4X7SKEQCXU665VXZLLQ4IF5I2S5CAY45IZSIUG5X4JKPZFA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.227923806 (240h28m16.04715983s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.327415309 (240h28m16.146651333s) [noise=true , address=81.171.14.70:4044 , piece_id=I2ML43UPUBAZWL6LNXEXOXVCO4VMHQULA6LIZ6QWSDES4DC4RMYA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:30.946412145 (240h28m17.765648159s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.310420672 (240h28m18.129656686s) [noise=true , address=89.149.201.147:4018 , piece_id=QQ4VAJ2AKLWWRV4FSPUMYB4CW64GZNVDBCHHF2FLLPUFKFOELHMA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.578914230 (240h28m18.398150274s) [noise=true , address=81.171.14.70:4030 , piece_id=RGRX2LD7VJUKZUI7OTHL54N4TCX6BT27SG2656CHNK3C4USIM3XA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:32.696287178 (240h28m19.515523202s) [noise=true , address=89.149.201.147:4004 , piece_id=YYIMUGDQSCZBIYNQRDQHY3FCHGDUQ7NYMD2P77N2KJTMHVCC5NLA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.122355826 (240h28m19.94159185s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622613706 (240h28m20.44184973s) [noise=true , address=81.171.14.70:4045 , piece_id=OXF6AHOJRIBSLIM2AHDMWHJELCXJJOK4FDUCZX4R2IWKSFIT4GTQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622631460 (240h28m20.441867484s) [seg_pos={0 0}, upload=230887]: Satisfied cohort requirements after 26 pieces
Apr 17 16:14:33.622880831 (240h28m20.442116865s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC): protocol: read tcp 188.166.225.170:33320->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622889879 (240h28m20.442125913s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622919847 (240h28m20.442155871s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jcAHt22bETT27CCXJ8ZaaYxxoKBonBYsFcoq6A34DVVjvhQAN): protocol: read tcp 188.166.225.170:44412->81.171.14.70:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622924756 (240h28m20.44216078s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622936359 (240h28m20.442172383s) [seg_pos={0 0}, upload=230887]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 17 16:14:33.622990012 (240h28m20.442226036s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 17 16:14:33.622993769 (240h28m20.442229793s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 17 16:14:33.857773622 (240h28m20.677009646s) [upload=230887]: Done uploading segments...
Apr 17 16:14:33.857791256 (240h28m20.67702728s) [upload=230887]: Upload finished. err: <nil>
Duration: 3.657076838s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
6.949936165s trace:
https://jaeger.dev.storj.io/trace/1c80c1acf9bc1e4e
FAIL (2026-04-17 16:49:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.949959368s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
6.721893338s trace:
https://jaeger.dev.storj.io/trace/1c80c1acf9bc1e4e
FAIL (2026-04-17 16:48:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.721916372s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
4.146743962s trace:
https://jaeger.dev.storj.io/trace/7f8a8e8c290cfce4
FAIL (2026-04-17 16:48:38 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: context deadline exceeded
======LOGS======
}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468778967 (240h28m13.288014991s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468982109 (240h28m13.288218133s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469034200 (240h28m13.288270224s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469278090 (240h28m13.288514114s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469317958 (240h28m13.288553982s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469533453 (240h28m13.288769477s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469783256 (240h28m13.28901928s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470149613 (240h28m13.289385637s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470407480 (240h28m13.289643504s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:27.287464596 (240h28m14.10670062s) [upload=230887]: Next returned nil segment
Apr 17 16:14:27.287476399 (240h28m14.106712423s) [upload=230887]: Waiting for error group managing segments...
Apr 17 16:14:27.667935875 (240h28m14.487171899s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.669691854 (240h28m14.488927878s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.672696102 (240h28m14.491932116s) [noise=true , address=185.167.204.206:4044 , piece_id=PV5G7HUDQCIZRBRC4U4QBSXDPYMS6OKELUO6T376WSCHIJEKSSHA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.673873083 (240h28m14.493109108s) [noise=true , address=185.167.204.205:4014 , piece_id=NZ2QB3PRHEREH7L3JN6QWWKEUSCCZJPX4KX5TOBBTEFI7V77CLDQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674467470 (240h28m14.493703494s) [noise=true , address=185.167.204.206:4041 , piece_id=5KYRFUXZKMCXVC6I2XF3MWFFFCBKRDEQU32J46PELU2K7ITHDJCQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674861982 (240h28m14.494098006s) [noise=true , address=185.167.204.205:4023 , piece_id=GX2SDFS6H5G3OV5K3JPMX4AFDJJWQAZZMJWCFCDLIQJ6J2EEGBRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675219832 (240h28m14.494455856s) [noise=true , address=185.167.204.206:4062 , piece_id=JLJJYF2HITUHLDWANNO7TANPR2B55XBEPI73RGIEQWQAUXVAUOLQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675769814 (240h28m14.495005838s) [noise=true , address=185.167.204.205:4034 , piece_id=2YLRAMZ4EUMTBIAOP47JESS2CNUE72WD2X7WNS3G5SNXTSIKZE6Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.679234931 (240h28m14.498470955s) [noise=true , address=185.167.204.205:4005 , piece_id=K5BMMNJJLPMJJ37OERBC7A2Y2GS6O7M5SKYFQSVUHWFWFQVC52SQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.680568836 (240h28m14.49980485s) [noise=true , address=185.167.204.205:4033 , piece_id=BVOQ7VXKB65CRU3M3APCYJ3NKFG6HG6UGNNU7UKI4CJYCYL5HY2Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.681606178 (240h28m14.500842202s) [noise=true , address=185.167.204.205:4004 , piece_id=UATYU6M64CYLYFO7ZGZZTMY6TUJM34WGYM5U4FKE2EADZ3WFHHXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.682849166 (240h28m14.50208519s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.683007592 (240h28m14.502243616s) [noise=true , address=185.167.204.205:4003 , piece_id=OAJS25SOIZDYBJK43WRYDDMO3OYR6P6NCE7O37KN7X2PICJNOZXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.684400130 (240h28m14.503636154s) [noise=true , address=185.167.204.205:4036 , piece_id=RMXRKSJM5DRMODM2CAZBXTKUMFTG4B4O26TMKC233WIOTRSR6KRQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.686095452 (240h28m14.505331466s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.747734833 (240h28m14.566970857s) [noise=true , address=81.171.14.70:4046 , piece_id=4NMIQBBHLZX3AZRNTOHSJ2LM5A26OV66FL7HENHJTGJPFZPDZO4A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:28.990034383 (240h28m15.809270407s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.104032302 (240h28m15.923268326s) [noise=true , address=89.149.201.147:4011 , piece_id=VJZIZ4X7SKEQCXU665VXZLLQ4IF5I2S5CAY45IZSIUG5X4JKPZFA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.227923806 (240h28m16.04715983s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.327415309 (240h28m16.146651333s) [noise=true , address=81.171.14.70:4044 , piece_id=I2ML43UPUBAZWL6LNXEXOXVCO4VMHQULA6LIZ6QWSDES4DC4RMYA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:30.946412145 (240h28m17.765648159s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.310420672 (240h28m18.129656686s) [noise=true , address=89.149.201.147:4018 , piece_id=QQ4VAJ2AKLWWRV4FSPUMYB4CW64GZNVDBCHHF2FLLPUFKFOELHMA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.578914230 (240h28m18.398150274s) [noise=true , address=81.171.14.70:4030 , piece_id=RGRX2LD7VJUKZUI7OTHL54N4TCX6BT27SG2656CHNK3C4USIM3XA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:32.696287178 (240h28m19.515523202s) [noise=true , address=89.149.201.147:4004 , piece_id=YYIMUGDQSCZBIYNQRDQHY3FCHGDUQ7NYMD2P77N2KJTMHVCC5NLA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.122355826 (240h28m19.94159185s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622613706 (240h28m20.44184973s) [noise=true , address=81.171.14.70:4045 , piece_id=OXF6AHOJRIBSLIM2AHDMWHJELCXJJOK4FDUCZX4R2IWKSFIT4GTQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622631460 (240h28m20.441867484s) [seg_pos={0 0}, upload=230887]: Satisfied cohort requirements after 26 pieces
Apr 17 16:14:33.622880831 (240h28m20.442116865s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC): protocol: read tcp 188.166.225.170:33320->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622889879 (240h28m20.442125913s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622919847 (240h28m20.442155871s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jcAHt22bETT27CCXJ8ZaaYxxoKBonBYsFcoq6A34DVVjvhQAN): protocol: read tcp 188.166.225.170:44412->81.171.14.70:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622924756 (240h28m20.44216078s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622936359 (240h28m20.442172383s) [seg_pos={0 0}, upload=230887]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 17 16:14:33.622990012 (240h28m20.442226036s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 17 16:14:33.622993769 (240h28m20.442229793s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 17 16:14:33.857773622 (240h28m20.677009646s) [upload=230887]: Done uploading segments...
Apr 17 16:14:33.857791256 (240h28m20.67702728s) [upload=230887]: Upload finished. err: <nil>
Duration: 4.146772729s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
4.209873188s trace:
https://jaeger.dev.storj.io/trace/7f8a8e8c290cfce4
FAIL (2026-04-17 16:48:05 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: context deadline exceeded
======LOGS======
}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468778967 (240h28m13.288014991s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.468982109 (240h28m13.288218133s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469034200 (240h28m13.288270224s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469278090 (240h28m13.288514114s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469317958 (240h28m13.288553982s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469533453 (240h28m13.288769477s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.469783256 (240h28m13.28901928s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470149613 (240h28m13.289385637s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:26.470407480 (240h28m13.289643504s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Uploading piece...
Apr 17 16:14:27.287464596 (240h28m14.10670062s) [upload=230887]: Next returned nil segment
Apr 17 16:14:27.287476399 (240h28m14.106712423s) [upload=230887]: Waiting for error group managing segments...
Apr 17 16:14:27.667935875 (240h28m14.487171899s) [noise=true , address=185.167.204.205:4030 , piece_id=COML2QVLEHIDERU5TATMOQYMQGMY4YRL7TJJHY6NQL7ZFKXEVJIQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.669691854 (240h28m14.488927878s) [noise=true , address=185.167.204.206:4064 , piece_id=KNUXG6ZYQOZJANMWLRLCNRBHHZDTAKJPPL7YM6IUW4LYOOI5ZM6A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.672696102 (240h28m14.491932116s) [noise=true , address=185.167.204.206:4044 , piece_id=PV5G7HUDQCIZRBRC4U4QBSXDPYMS6OKELUO6T376WSCHIJEKSSHA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.673873083 (240h28m14.493109108s) [noise=true , address=185.167.204.205:4014 , piece_id=NZ2QB3PRHEREH7L3JN6QWWKEUSCCZJPX4KX5TOBBTEFI7V77CLDQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674467470 (240h28m14.493703494s) [noise=true , address=185.167.204.206:4041 , piece_id=5KYRFUXZKMCXVC6I2XF3MWFFFCBKRDEQU32J46PELU2K7ITHDJCQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.674861982 (240h28m14.494098006s) [noise=true , address=185.167.204.205:4023 , piece_id=GX2SDFS6H5G3OV5K3JPMX4AFDJJWQAZZMJWCFCDLIQJ6J2EEGBRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675219832 (240h28m14.494455856s) [noise=true , address=185.167.204.206:4062 , piece_id=JLJJYF2HITUHLDWANNO7TANPR2B55XBEPI73RGIEQWQAUXVAUOLQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.675769814 (240h28m14.495005838s) [noise=true , address=185.167.204.205:4034 , piece_id=2YLRAMZ4EUMTBIAOP47JESS2CNUE72WD2X7WNS3G5SNXTSIKZE6Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.679234931 (240h28m14.498470955s) [noise=true , address=185.167.204.205:4005 , piece_id=K5BMMNJJLPMJJ37OERBC7A2Y2GS6O7M5SKYFQSVUHWFWFQVC52SQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.680568836 (240h28m14.49980485s) [noise=true , address=185.167.204.205:4033 , piece_id=BVOQ7VXKB65CRU3M3APCYJ3NKFG6HG6UGNNU7UKI4CJYCYL5HY2Q, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.681606178 (240h28m14.500842202s) [noise=true , address=185.167.204.205:4004 , piece_id=UATYU6M64CYLYFO7ZGZZTMY6TUJM34WGYM5U4FKE2EADZ3WFHHXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.682849166 (240h28m14.50208519s) [noise=true , address=185.167.204.205:4013 , piece_id=2NUCNBBKMTID2O3QR5TTOISZZSXQICSHXUBNY3GGHCYBD4UDLJHQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.683007592 (240h28m14.502243616s) [noise=true , address=185.167.204.205:4003 , piece_id=OAJS25SOIZDYBJK43WRYDDMO3OYR6P6NCE7O37KN7X2PICJNOZXA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.684400130 (240h28m14.503636154s) [noise=true , address=185.167.204.205:4036 , piece_id=RMXRKSJM5DRMODM2CAZBXTKUMFTG4B4O26TMKC233WIOTRSR6KRQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.686095452 (240h28m14.505331466s) [noise=true , address=185.167.204.206:4046 , piece_id=MMAOAJI57UPP6GCOU6TSBWS2QLKRPJQZXOBBI4XFJ5JBM237FOMQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:27.747734833 (240h28m14.566970857s) [noise=true , address=81.171.14.70:4046 , piece_id=4NMIQBBHLZX3AZRNTOHSJ2LM5A26OV66FL7HENHJTGJPFZPDZO4A, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:28.990034383 (240h28m15.809270407s) [noise=true , address=81.171.14.70:4047 , piece_id=G26XGG4KY3Z6MTRVEHCTMCJRKJEKONHHCOCRVMJBBR5DRWXDXUQQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.104032302 (240h28m15.923268326s) [noise=true , address=89.149.201.147:4011 , piece_id=VJZIZ4X7SKEQCXU665VXZLLQ4IF5I2S5CAY45IZSIUG5X4JKPZFA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.227923806 (240h28m16.04715983s) [noise=true , address=89.149.201.147:4010 , piece_id=YHLIRLUQAK2IEKDWTGVAUBWVT2IROVTZWDLXRR5MQTA5GJJ4UBAA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:29.327415309 (240h28m16.146651333s) [noise=true , address=81.171.14.70:4044 , piece_id=I2ML43UPUBAZWL6LNXEXOXVCO4VMHQULA6LIZ6QWSDES4DC4RMYA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:30.946412145 (240h28m17.765648159s) [noise=true , address=89.149.201.147:4019 , piece_id=WVD53JIIOO4U5UZPKCR3HBYZJ4N7HEZY5B2MGFSO2M4PCFJN52VQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.310420672 (240h28m18.129656686s) [noise=true , address=89.149.201.147:4018 , piece_id=QQ4VAJ2AKLWWRV4FSPUMYB4CW64GZNVDBCHHF2FLLPUFKFOELHMA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:31.578914230 (240h28m18.398150274s) [noise=true , address=81.171.14.70:4030 , piece_id=RGRX2LD7VJUKZUI7OTHL54N4TCX6BT27SG2656CHNK3C4USIM3XA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:32.696287178 (240h28m19.515523202s) [noise=true , address=89.149.201.147:4004 , piece_id=YYIMUGDQSCZBIYNQRDQHY3FCHGDUQ7NYMD2P77N2KJTMHVCC5NLA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.122355826 (240h28m19.94159185s) [noise=true , address=81.171.14.70:4033 , piece_id=DGEEAWVRR5PDWBXYRUY4FXZPG7BUWMJQP5LTPBF3HAKDOQGDVESA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622613706 (240h28m20.44184973s) [noise=true , address=81.171.14.70:4045 , piece_id=OXF6AHOJRIBSLIM2AHDMWHJELCXJJOK4FDUCZX4R2IWKSFIT4GTQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: <nil>
Apr 17 16:14:33.622631460 (240h28m20.441867484s) [seg_pos={0 0}, upload=230887]: Satisfied cohort requirements after 26 pieces
Apr 17 16:14:33.622880831 (240h28m20.442116865s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U7CWC6tPDsNkmYf6uyKgFjwvYaUwUte6dmzUsZQPzm7ecPsLC): protocol: read tcp 188.166.225.170:33320->81.171.14.70:4041: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622889879 (240h28m20.442125913s) [noise=true , address=81.171.14.70:4041 , piece_id=EAEVASBBAGGP42TF7QWHXD3DYPGUOBRTSPRCAIF6WZE6DNEM2SJQ, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622919847 (240h28m20.442155871s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jcAHt22bETT27CCXJ8ZaaYxxoKBonBYsFcoq6A34DVVjvhQAN): protocol: read tcp 188.166.225.170:44412->81.171.14.70:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 17 16:14:33.622924756 (240h28m20.44216078s) [noise=true , address=81.171.14.70:4034 , piece_id=H5Y65EJCRSDH4PHJSYEM3HMLGDFMAN357SJEIXDWR2HQWRMHQDRA, seg_pos={0 0}, upload=230887]: Long tail cancellation due to optimal threshold
Apr 17 16:14:33.622936359 (240h28m20.442172383s) [seg_pos={0 0}, upload=230887]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Apr 17 16:14:33.622990012 (240h28m20.442226036s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Apr 17 16:14:33.622993769 (240h28m20.442229793s) [upload=230887]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Apr 17 16:14:33.857773622 (240h28m20.677009646s) [upload=230887]: Done uploading segments...
Apr 17 16:14:33.857791256 (240h28m20.67702728s) [upload=230887]: Upload finished. err: <nil>
Duration: 4.20990077s
core
uplink
eu-select-1
eu1-placement-15
|