Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 28 out of 60 is failed (threshold=6) [.X..X.X...X.X......XXXX.XX..XXXXXXXX.XXXXX......X......XX.X.] duration: 5.001099591s trace: https://jaeger.dev.storj.io/trace/1fbaf1925b8c2bee FAIL (2026-04-17 23:02:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001126753s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [X.....X...X...X........XX...................................] duration: 8.07303765s trace: https://jaeger.dev.storj.io/trace/1e79bf0cbb46f7f9 FAIL (2026-04-17 23:02:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ments... Apr 17 22:37:58.471270102 (246h51m45.290506126s) [upload=237005]: Got next segment. Inline: false Apr 17 22:37:58.471378241 (246h51m45.290614265s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 17 22:37:58.471387018 (246h51m45.290623052s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 17 22:37:58.708193142 (246h51m45.527429166s) [seg_pos={0 0}, upload=237005]: Begin upload segment... Apr 17 22:37:58.708247858 (246h51m45.527483882s) [seg_pos={0 0}, upload=237005]: Done begin upload segment. Apr 17 22:37:58.708272956 (246h51m45.52750898s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708361497 (246h51m45.527597561s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708820833 (246h51m45.528056857s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708977306 (246h51m45.52821333s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709119320 (246h51m45.528355344s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709308925 (246h51m45.52854495s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709392147 (246h51m45.528628171s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709605307 (246h51m45.528841331s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709666606 (246h51m45.52890263s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709922579 (246h51m45.529158603s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709971264 (246h51m45.529207288s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710303314 (246h51m45.529539338s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710310148 (246h51m45.529546172s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710527357 (246h51m45.529763381s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710597883 (246h51m45.529833907s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710890227 (246h51m45.530126251s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:38:00.506886217 (246h51m47.326122241s) [upload=237005]: Next returned nil segment Apr 17 22:38:00.506910955 (246h51m47.326146989s) [upload=237005]: Waiting for error group managing segments... Apr 17 22:38:00.795192183 (246h51m47.614428217s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.802966977 (246h51m47.622203012s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.837150420 (246h51m47.656386444s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.857273577 (246h51m47.676509601s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:01.831163825 (246h51m48.650399849s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.129931912 (246h51m48.949167936s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.167981246 (246h51m48.98721728s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.179145783 (246h51m48.998381807s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.275673416 (246h51m49.09490945s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.654733063 (246h51m49.473969097s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.733948735 (246h51m49.553184759s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.806700572 (246h51m49.625936596s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807972947 (246h51m49.627208972s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807993647 (246h51m49.627229681s) [seg_pos={0 0}, upload=237005]: Satisfied cohort requirements after 13 pieces Apr 17 22:38:02.808282966 (246h51m49.62751901s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: expected piece hash; context canceled; EOF Apr 17 22:38:02.808293136 (246h51m49.62752917s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808429940 (246h51m49.627665964s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection Apr 17 22:38:02.808441132 (246h51m49.627677166s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808538750 (246h51m49.627774784s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QtY3yN6J1H8htRneePHu4RmbN7A1xkgywphMDVsVxhTczGupY): protocol: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection Apr 17 22:38:02.808549581 (246h51m49.627785615s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808579067 (246h51m49.627815091s) [seg_pos={0 0}, upload=237005]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 17 22:38:02.808667418 (246h51m49.627903452s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 17 22:38:02.808674110 (246h51m49.627910134s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 17 22:38:03.086673887 (246h51m49.905909911s) [upload=237005]: Done uploading segments... Apr 17 22:38:03.086701991 (246h51m49.905938015s) [upload=237005]: Upload finished. err: <nil> Duration: 8.073073529s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.X.....X...X...X........XX..................................] duration: 8.145058502s trace: https://jaeger.dev.storj.io/trace/1e79bf0cbb46f7f9 FAIL (2026-04-17 23:01:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ments... Apr 17 22:37:58.471270102 (246h51m45.290506126s) [upload=237005]: Got next segment. Inline: false Apr 17 22:37:58.471378241 (246h51m45.290614265s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 17 22:37:58.471387018 (246h51m45.290623052s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 17 22:37:58.708193142 (246h51m45.527429166s) [seg_pos={0 0}, upload=237005]: Begin upload segment... Apr 17 22:37:58.708247858 (246h51m45.527483882s) [seg_pos={0 0}, upload=237005]: Done begin upload segment. Apr 17 22:37:58.708272956 (246h51m45.52750898s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708361497 (246h51m45.527597561s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708820833 (246h51m45.528056857s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708977306 (246h51m45.52821333s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709119320 (246h51m45.528355344s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709308925 (246h51m45.52854495s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709392147 (246h51m45.528628171s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709605307 (246h51m45.528841331s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709666606 (246h51m45.52890263s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709922579 (246h51m45.529158603s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709971264 (246h51m45.529207288s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710303314 (246h51m45.529539338s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710310148 (246h51m45.529546172s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710527357 (246h51m45.529763381s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710597883 (246h51m45.529833907s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710890227 (246h51m45.530126251s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:38:00.506886217 (246h51m47.326122241s) [upload=237005]: Next returned nil segment Apr 17 22:38:00.506910955 (246h51m47.326146989s) [upload=237005]: Waiting for error group managing segments... Apr 17 22:38:00.795192183 (246h51m47.614428217s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.802966977 (246h51m47.622203012s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.837150420 (246h51m47.656386444s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.857273577 (246h51m47.676509601s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:01.831163825 (246h51m48.650399849s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.129931912 (246h51m48.949167936s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.167981246 (246h51m48.98721728s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.179145783 (246h51m48.998381807s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.275673416 (246h51m49.09490945s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.654733063 (246h51m49.473969097s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.733948735 (246h51m49.553184759s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.806700572 (246h51m49.625936596s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807972947 (246h51m49.627208972s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807993647 (246h51m49.627229681s) [seg_pos={0 0}, upload=237005]: Satisfied cohort requirements after 13 pieces Apr 17 22:38:02.808282966 (246h51m49.62751901s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: expected piece hash; context canceled; EOF Apr 17 22:38:02.808293136 (246h51m49.62752917s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808429940 (246h51m49.627665964s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection Apr 17 22:38:02.808441132 (246h51m49.627677166s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808538750 (246h51m49.627774784s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QtY3yN6J1H8htRneePHu4RmbN7A1xkgywphMDVsVxhTczGupY): protocol: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection Apr 17 22:38:02.808549581 (246h51m49.627785615s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808579067 (246h51m49.627815091s) [seg_pos={0 0}, upload=237005]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 17 22:38:02.808667418 (246h51m49.627903452s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 17 22:38:02.808674110 (246h51m49.627910134s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 17 22:38:03.086673887 (246h51m49.905909911s) [upload=237005]: Done uploading segments... Apr 17 22:38:03.086701991 (246h51m49.905938015s) [upload=237005]: Upload finished. err: <nil> Duration: 8.145082919s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 10 out of 60 is failed (threshold=3) [........XXXXXXXXXX..........................................] duration: 7.457925132s trace: https://jaeger.dev.storj.io/trace/51ac1c35b102be61 FAIL (2026-04-17 23:01:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.45796022s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 28 out of 60 is failed (threshold=6) [..X..X.X...X.X......XXXX.XX..XXXXXXXX.XXXXX......X......XX.X] duration: 5.001140498s trace: https://jaeger.dev.storj.io/trace/1fbaf1925b8c2bee FAIL (2026-04-17 23:01:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001165747s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..X.....X...X...X........XX.................................] duration: 8.934369584s trace: https://jaeger.dev.storj.io/trace/1e79bf0cbb46f7f9 FAIL (2026-04-17 23:01:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ments... Apr 17 22:37:58.471270102 (246h51m45.290506126s) [upload=237005]: Got next segment. Inline: false Apr 17 22:37:58.471378241 (246h51m45.290614265s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 17 22:37:58.471387018 (246h51m45.290623052s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 17 22:37:58.708193142 (246h51m45.527429166s) [seg_pos={0 0}, upload=237005]: Begin upload segment... Apr 17 22:37:58.708247858 (246h51m45.527483882s) [seg_pos={0 0}, upload=237005]: Done begin upload segment. Apr 17 22:37:58.708272956 (246h51m45.52750898s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708361497 (246h51m45.527597561s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708820833 (246h51m45.528056857s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708977306 (246h51m45.52821333s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709119320 (246h51m45.528355344s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709308925 (246h51m45.52854495s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709392147 (246h51m45.528628171s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709605307 (246h51m45.528841331s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709666606 (246h51m45.52890263s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709922579 (246h51m45.529158603s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709971264 (246h51m45.529207288s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710303314 (246h51m45.529539338s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710310148 (246h51m45.529546172s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710527357 (246h51m45.529763381s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710597883 (246h51m45.529833907s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710890227 (246h51m45.530126251s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:38:00.506886217 (246h51m47.326122241s) [upload=237005]: Next returned nil segment Apr 17 22:38:00.506910955 (246h51m47.326146989s) [upload=237005]: Waiting for error group managing segments... Apr 17 22:38:00.795192183 (246h51m47.614428217s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.802966977 (246h51m47.622203012s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.837150420 (246h51m47.656386444s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.857273577 (246h51m47.676509601s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:01.831163825 (246h51m48.650399849s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.129931912 (246h51m48.949167936s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.167981246 (246h51m48.98721728s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.179145783 (246h51m48.998381807s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.275673416 (246h51m49.09490945s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.654733063 (246h51m49.473969097s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.733948735 (246h51m49.553184759s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.806700572 (246h51m49.625936596s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807972947 (246h51m49.627208972s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807993647 (246h51m49.627229681s) [seg_pos={0 0}, upload=237005]: Satisfied cohort requirements after 13 pieces Apr 17 22:38:02.808282966 (246h51m49.62751901s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: expected piece hash; context canceled; EOF Apr 17 22:38:02.808293136 (246h51m49.62752917s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808429940 (246h51m49.627665964s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection Apr 17 22:38:02.808441132 (246h51m49.627677166s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808538750 (246h51m49.627774784s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QtY3yN6J1H8htRneePHu4RmbN7A1xkgywphMDVsVxhTczGupY): protocol: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection Apr 17 22:38:02.808549581 (246h51m49.627785615s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808579067 (246h51m49.627815091s) [seg_pos={0 0}, upload=237005]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 17 22:38:02.808667418 (246h51m49.627903452s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 17 22:38:02.808674110 (246h51m49.627910134s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 17 22:38:03.086673887 (246h51m49.905909911s) [upload=237005]: Done uploading segments... Apr 17 22:38:03.086701991 (246h51m49.905938015s) [upload=237005]: Upload finished. err: <nil> Duration: 8.934404471s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 27 out of 60 is failed (threshold=6) [...X..X.X...X.X......XXXX.XX..XXXXXXXX.XXXXX......X......XX.] duration: 3.73051575s trace: https://jaeger.dev.storj.io/trace/15d94dcc91203af FAIL (2026-04-17 23:01:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.730544576s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...X.....X...X...X........XX................................] duration: 6.964352895s trace: https://jaeger.dev.storj.io/trace/1e79bf0cbb46f7f9 FAIL (2026-04-17 23:00:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ments... Apr 17 22:37:58.471270102 (246h51m45.290506126s) [upload=237005]: Got next segment. Inline: false Apr 17 22:37:58.471378241 (246h51m45.290614265s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 17 22:37:58.471387018 (246h51m45.290623052s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 17 22:37:58.708193142 (246h51m45.527429166s) [seg_pos={0 0}, upload=237005]: Begin upload segment... Apr 17 22:37:58.708247858 (246h51m45.527483882s) [seg_pos={0 0}, upload=237005]: Done begin upload segment. Apr 17 22:37:58.708272956 (246h51m45.52750898s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708361497 (246h51m45.527597561s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708820833 (246h51m45.528056857s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708977306 (246h51m45.52821333s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709119320 (246h51m45.528355344s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709308925 (246h51m45.52854495s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709392147 (246h51m45.528628171s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709605307 (246h51m45.528841331s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709666606 (246h51m45.52890263s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709922579 (246h51m45.529158603s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709971264 (246h51m45.529207288s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710303314 (246h51m45.529539338s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710310148 (246h51m45.529546172s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710527357 (246h51m45.529763381s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710597883 (246h51m45.529833907s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710890227 (246h51m45.530126251s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:38:00.506886217 (246h51m47.326122241s) [upload=237005]: Next returned nil segment Apr 17 22:38:00.506910955 (246h51m47.326146989s) [upload=237005]: Waiting for error group managing segments... Apr 17 22:38:00.795192183 (246h51m47.614428217s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.802966977 (246h51m47.622203012s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.837150420 (246h51m47.656386444s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.857273577 (246h51m47.676509601s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:01.831163825 (246h51m48.650399849s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.129931912 (246h51m48.949167936s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.167981246 (246h51m48.98721728s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.179145783 (246h51m48.998381807s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.275673416 (246h51m49.09490945s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.654733063 (246h51m49.473969097s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.733948735 (246h51m49.553184759s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.806700572 (246h51m49.625936596s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807972947 (246h51m49.627208972s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807993647 (246h51m49.627229681s) [seg_pos={0 0}, upload=237005]: Satisfied cohort requirements after 13 pieces Apr 17 22:38:02.808282966 (246h51m49.62751901s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: expected piece hash; context canceled; EOF Apr 17 22:38:02.808293136 (246h51m49.62752917s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808429940 (246h51m49.627665964s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection Apr 17 22:38:02.808441132 (246h51m49.627677166s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808538750 (246h51m49.627774784s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QtY3yN6J1H8htRneePHu4RmbN7A1xkgywphMDVsVxhTczGupY): protocol: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection Apr 17 22:38:02.808549581 (246h51m49.627785615s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808579067 (246h51m49.627815091s) [seg_pos={0 0}, upload=237005]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 17 22:38:02.808667418 (246h51m49.627903452s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 17 22:38:02.808674110 (246h51m49.627910134s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 17 22:38:03.086673887 (246h51m49.905909911s) [upload=237005]: Done uploading segments... Apr 17 22:38:03.086701991 (246h51m49.905938015s) [upload=237005]: Upload finished. err: <nil> Duration: 6.964394064s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 10 out of 60 is failed (threshold=3) [.........XXXXXXXXXX.........................................] duration: 6.938817491s trace: https://jaeger.dev.storj.io/trace/51ac1c35b102be61 FAIL (2026-04-17 23:00:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.938845075s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [....X.....X...X...X........XX...............................] duration: 8.786138028s trace: https://jaeger.dev.storj.io/trace/1e79bf0cbb46f7f9 FAIL (2026-04-17 23:00:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ments... Apr 17 22:37:58.471270102 (246h51m45.290506126s) [upload=237005]: Got next segment. Inline: false Apr 17 22:37:58.471378241 (246h51m45.290614265s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 17 22:37:58.471387018 (246h51m45.290623052s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 17 22:37:58.708193142 (246h51m45.527429166s) [seg_pos={0 0}, upload=237005]: Begin upload segment... Apr 17 22:37:58.708247858 (246h51m45.527483882s) [seg_pos={0 0}, upload=237005]: Done begin upload segment. Apr 17 22:37:58.708272956 (246h51m45.52750898s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708361497 (246h51m45.527597561s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708820833 (246h51m45.528056857s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708977306 (246h51m45.52821333s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709119320 (246h51m45.528355344s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709308925 (246h51m45.52854495s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709392147 (246h51m45.528628171s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709605307 (246h51m45.528841331s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709666606 (246h51m45.52890263s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709922579 (246h51m45.529158603s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709971264 (246h51m45.529207288s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710303314 (246h51m45.529539338s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710310148 (246h51m45.529546172s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710527357 (246h51m45.529763381s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710597883 (246h51m45.529833907s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710890227 (246h51m45.530126251s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:38:00.506886217 (246h51m47.326122241s) [upload=237005]: Next returned nil segment Apr 17 22:38:00.506910955 (246h51m47.326146989s) [upload=237005]: Waiting for error group managing segments... Apr 17 22:38:00.795192183 (246h51m47.614428217s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.802966977 (246h51m47.622203012s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.837150420 (246h51m47.656386444s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.857273577 (246h51m47.676509601s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:01.831163825 (246h51m48.650399849s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.129931912 (246h51m48.949167936s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.167981246 (246h51m48.98721728s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.179145783 (246h51m48.998381807s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.275673416 (246h51m49.09490945s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.654733063 (246h51m49.473969097s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.733948735 (246h51m49.553184759s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.806700572 (246h51m49.625936596s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807972947 (246h51m49.627208972s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807993647 (246h51m49.627229681s) [seg_pos={0 0}, upload=237005]: Satisfied cohort requirements after 13 pieces Apr 17 22:38:02.808282966 (246h51m49.62751901s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: expected piece hash; context canceled; EOF Apr 17 22:38:02.808293136 (246h51m49.62752917s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808429940 (246h51m49.627665964s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection Apr 17 22:38:02.808441132 (246h51m49.627677166s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808538750 (246h51m49.627774784s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QtY3yN6J1H8htRneePHu4RmbN7A1xkgywphMDVsVxhTczGupY): protocol: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection Apr 17 22:38:02.808549581 (246h51m49.627785615s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808579067 (246h51m49.627815091s) [seg_pos={0 0}, upload=237005]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 17 22:38:02.808667418 (246h51m49.627903452s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 17 22:38:02.808674110 (246h51m49.627910134s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 17 22:38:03.086673887 (246h51m49.905909911s) [upload=237005]: Done uploading segments... Apr 17 22:38:03.086701991 (246h51m49.905938015s) [upload=237005]: Upload finished. err: <nil> Duration: 8.786168946s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 28 out of 60 is failed (threshold=6) [X...X..X.X...X.X......XXXX.XX..XXXXXXXX.XXXXX......X......XX] duration: 5.000706653s trace: https://jaeger.dev.storj.io/trace/15d94dcc91203af FAIL (2026-04-17 22:59:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.00073089s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 10 out of 60 is failed (threshold=3) [..........XXXXXXXXXX........................................] duration: 7.213442626s trace: https://jaeger.dev.storj.io/trace/51ac1c35b102be61 FAIL (2026-04-17 22:59:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.213464348s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....X.....X...X...X........XX..............................] duration: 7.501808324s trace: https://jaeger.dev.storj.io/trace/1e79bf0cbb46f7f9 FAIL (2026-04-17 22:59:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ments... Apr 17 22:37:58.471270102 (246h51m45.290506126s) [upload=237005]: Got next segment. Inline: false Apr 17 22:37:58.471378241 (246h51m45.290614265s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 17 22:37:58.471387018 (246h51m45.290623052s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 17 22:37:58.708193142 (246h51m45.527429166s) [seg_pos={0 0}, upload=237005]: Begin upload segment... Apr 17 22:37:58.708247858 (246h51m45.527483882s) [seg_pos={0 0}, upload=237005]: Done begin upload segment. Apr 17 22:37:58.708272956 (246h51m45.52750898s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708361497 (246h51m45.527597561s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708820833 (246h51m45.528056857s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708977306 (246h51m45.52821333s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709119320 (246h51m45.528355344s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709308925 (246h51m45.52854495s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709392147 (246h51m45.528628171s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709605307 (246h51m45.528841331s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709666606 (246h51m45.52890263s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709922579 (246h51m45.529158603s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709971264 (246h51m45.529207288s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710303314 (246h51m45.529539338s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710310148 (246h51m45.529546172s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710527357 (246h51m45.529763381s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710597883 (246h51m45.529833907s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710890227 (246h51m45.530126251s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:38:00.506886217 (246h51m47.326122241s) [upload=237005]: Next returned nil segment Apr 17 22:38:00.506910955 (246h51m47.326146989s) [upload=237005]: Waiting for error group managing segments... Apr 17 22:38:00.795192183 (246h51m47.614428217s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.802966977 (246h51m47.622203012s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.837150420 (246h51m47.656386444s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.857273577 (246h51m47.676509601s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:01.831163825 (246h51m48.650399849s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.129931912 (246h51m48.949167936s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.167981246 (246h51m48.98721728s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.179145783 (246h51m48.998381807s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.275673416 (246h51m49.09490945s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.654733063 (246h51m49.473969097s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.733948735 (246h51m49.553184759s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.806700572 (246h51m49.625936596s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807972947 (246h51m49.627208972s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807993647 (246h51m49.627229681s) [seg_pos={0 0}, upload=237005]: Satisfied cohort requirements after 13 pieces Apr 17 22:38:02.808282966 (246h51m49.62751901s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: expected piece hash; context canceled; EOF Apr 17 22:38:02.808293136 (246h51m49.62752917s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808429940 (246h51m49.627665964s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection Apr 17 22:38:02.808441132 (246h51m49.627677166s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808538750 (246h51m49.627774784s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QtY3yN6J1H8htRneePHu4RmbN7A1xkgywphMDVsVxhTczGupY): protocol: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection Apr 17 22:38:02.808549581 (246h51m49.627785615s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808579067 (246h51m49.627815091s) [seg_pos={0 0}, upload=237005]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 17 22:38:02.808667418 (246h51m49.627903452s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 17 22:38:02.808674110 (246h51m49.627910134s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 17 22:38:03.086673887 (246h51m49.905909911s) [upload=237005]: Done uploading segments... Apr 17 22:38:03.086701991 (246h51m49.905938015s) [upload=237005]: Upload finished. err: <nil> Duration: 7.501833373s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [......X.....X...X...X........XX.............................] duration: 7.54009904s trace: https://jaeger.dev.storj.io/trace/1e79bf0cbb46f7f9 FAIL (2026-04-17 22:59:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ments... Apr 17 22:37:58.471270102 (246h51m45.290506126s) [upload=237005]: Got next segment. Inline: false Apr 17 22:37:58.471378241 (246h51m45.290614265s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 17 22:37:58.471387018 (246h51m45.290623052s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 17 22:37:58.708193142 (246h51m45.527429166s) [seg_pos={0 0}, upload=237005]: Begin upload segment... Apr 17 22:37:58.708247858 (246h51m45.527483882s) [seg_pos={0 0}, upload=237005]: Done begin upload segment. Apr 17 22:37:58.708272956 (246h51m45.52750898s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708361497 (246h51m45.527597561s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708820833 (246h51m45.528056857s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708977306 (246h51m45.52821333s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709119320 (246h51m45.528355344s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709308925 (246h51m45.52854495s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709392147 (246h51m45.528628171s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709605307 (246h51m45.528841331s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709666606 (246h51m45.52890263s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709922579 (246h51m45.529158603s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709971264 (246h51m45.529207288s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710303314 (246h51m45.529539338s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710310148 (246h51m45.529546172s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710527357 (246h51m45.529763381s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710597883 (246h51m45.529833907s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710890227 (246h51m45.530126251s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:38:00.506886217 (246h51m47.326122241s) [upload=237005]: Next returned nil segment Apr 17 22:38:00.506910955 (246h51m47.326146989s) [upload=237005]: Waiting for error group managing segments... Apr 17 22:38:00.795192183 (246h51m47.614428217s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.802966977 (246h51m47.622203012s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.837150420 (246h51m47.656386444s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.857273577 (246h51m47.676509601s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:01.831163825 (246h51m48.650399849s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.129931912 (246h51m48.949167936s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.167981246 (246h51m48.98721728s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.179145783 (246h51m48.998381807s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.275673416 (246h51m49.09490945s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.654733063 (246h51m49.473969097s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.733948735 (246h51m49.553184759s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.806700572 (246h51m49.625936596s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807972947 (246h51m49.627208972s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807993647 (246h51m49.627229681s) [seg_pos={0 0}, upload=237005]: Satisfied cohort requirements after 13 pieces Apr 17 22:38:02.808282966 (246h51m49.62751901s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: expected piece hash; context canceled; EOF Apr 17 22:38:02.808293136 (246h51m49.62752917s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808429940 (246h51m49.627665964s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection Apr 17 22:38:02.808441132 (246h51m49.627677166s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808538750 (246h51m49.627774784s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QtY3yN6J1H8htRneePHu4RmbN7A1xkgywphMDVsVxhTczGupY): protocol: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection Apr 17 22:38:02.808549581 (246h51m49.627785615s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808579067 (246h51m49.627815091s) [seg_pos={0 0}, upload=237005]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 17 22:38:02.808667418 (246h51m49.627903452s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 17 22:38:02.808674110 (246h51m49.627910134s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 17 22:38:03.086673887 (246h51m49.905909911s) [upload=237005]: Done uploading segments... Apr 17 22:38:03.086701991 (246h51m49.905938015s) [upload=237005]: Upload finished. err: <nil> Duration: 7.54011935s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 28 out of 60 is failed (threshold=6) [XX...X..X.X...X.X......XXXX.XX..XXXXXXXX.XXXXX......X......X] duration: 5.000807327s trace: https://jaeger.dev.storj.io/trace/1571c5835ba2a32a FAIL (2026-04-17 22:59:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000830061s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 10 out of 60 is failed (threshold=3) [...........XXXXXXXXXX.......................................] duration: 11.671438253s trace: https://jaeger.dev.storj.io/trace/51ac1c35b102be61 FAIL (2026-04-17 22:59:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 11.671521994s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.......X.....X...X...X........XX............................] duration: 19.189258611s trace: https://jaeger.dev.storj.io/trace/1e79bf0cbb46f7f9 FAIL (2026-04-17 22:58:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ments... Apr 17 22:37:58.471270102 (246h51m45.290506126s) [upload=237005]: Got next segment. Inline: false Apr 17 22:37:58.471378241 (246h51m45.290614265s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 17 22:37:58.471387018 (246h51m45.290623052s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 17 22:37:58.708193142 (246h51m45.527429166s) [seg_pos={0 0}, upload=237005]: Begin upload segment... Apr 17 22:37:58.708247858 (246h51m45.527483882s) [seg_pos={0 0}, upload=237005]: Done begin upload segment. Apr 17 22:37:58.708272956 (246h51m45.52750898s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708361497 (246h51m45.527597561s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708820833 (246h51m45.528056857s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708977306 (246h51m45.52821333s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709119320 (246h51m45.528355344s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709308925 (246h51m45.52854495s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709392147 (246h51m45.528628171s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709605307 (246h51m45.528841331s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709666606 (246h51m45.52890263s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709922579 (246h51m45.529158603s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709971264 (246h51m45.529207288s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710303314 (246h51m45.529539338s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710310148 (246h51m45.529546172s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710527357 (246h51m45.529763381s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710597883 (246h51m45.529833907s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710890227 (246h51m45.530126251s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:38:00.506886217 (246h51m47.326122241s) [upload=237005]: Next returned nil segment Apr 17 22:38:00.506910955 (246h51m47.326146989s) [upload=237005]: Waiting for error group managing segments... Apr 17 22:38:00.795192183 (246h51m47.614428217s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.802966977 (246h51m47.622203012s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.837150420 (246h51m47.656386444s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.857273577 (246h51m47.676509601s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:01.831163825 (246h51m48.650399849s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.129931912 (246h51m48.949167936s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.167981246 (246h51m48.98721728s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.179145783 (246h51m48.998381807s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.275673416 (246h51m49.09490945s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.654733063 (246h51m49.473969097s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.733948735 (246h51m49.553184759s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.806700572 (246h51m49.625936596s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807972947 (246h51m49.627208972s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807993647 (246h51m49.627229681s) [seg_pos={0 0}, upload=237005]: Satisfied cohort requirements after 13 pieces Apr 17 22:38:02.808282966 (246h51m49.62751901s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: expected piece hash; context canceled; EOF Apr 17 22:38:02.808293136 (246h51m49.62752917s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808429940 (246h51m49.627665964s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection Apr 17 22:38:02.808441132 (246h51m49.627677166s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808538750 (246h51m49.627774784s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QtY3yN6J1H8htRneePHu4RmbN7A1xkgywphMDVsVxhTczGupY): protocol: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection Apr 17 22:38:02.808549581 (246h51m49.627785615s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808579067 (246h51m49.627815091s) [seg_pos={0 0}, upload=237005]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 17 22:38:02.808667418 (246h51m49.627903452s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 17 22:38:02.808674110 (246h51m49.627910134s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 17 22:38:03.086673887 (246h51m49.905909911s) [upload=237005]: Done uploading segments... Apr 17 22:38:03.086701991 (246h51m49.905938015s) [upload=237005]: Upload finished. err: <nil> Duration: 19.189274561s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 28 out of 60 is failed (threshold=6) [XXX...X..X.X...X.X......XXXX.XX..XXXXXXXX.XXXXX......X......] duration: 4.463272303s trace: https://jaeger.dev.storj.io/trace/139507385cb7d1ef FAIL (2026-04-17 22:58:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.463299796s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 10 out of 60 is failed (threshold=3) [............XXXXXXXXXX......................................] duration: 9.195861738s trace: https://jaeger.dev.storj.io/trace/51ac1c35b102be61 FAIL (2026-04-17 22:58:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 9.195891546s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [........X.....X...X...X........XX...........................] duration: 7.472972729s trace: https://jaeger.dev.storj.io/trace/1e79bf0cbb46f7f9 FAIL (2026-04-17 22:57:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ments... Apr 17 22:37:58.471270102 (246h51m45.290506126s) [upload=237005]: Got next segment. Inline: false Apr 17 22:37:58.471378241 (246h51m45.290614265s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 17 22:37:58.471387018 (246h51m45.290623052s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 17 22:37:58.708193142 (246h51m45.527429166s) [seg_pos={0 0}, upload=237005]: Begin upload segment... Apr 17 22:37:58.708247858 (246h51m45.527483882s) [seg_pos={0 0}, upload=237005]: Done begin upload segment. Apr 17 22:37:58.708272956 (246h51m45.52750898s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708361497 (246h51m45.527597561s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708820833 (246h51m45.528056857s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708977306 (246h51m45.52821333s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709119320 (246h51m45.528355344s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709308925 (246h51m45.52854495s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709392147 (246h51m45.528628171s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709605307 (246h51m45.528841331s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709666606 (246h51m45.52890263s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709922579 (246h51m45.529158603s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709971264 (246h51m45.529207288s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710303314 (246h51m45.529539338s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710310148 (246h51m45.529546172s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710527357 (246h51m45.529763381s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710597883 (246h51m45.529833907s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710890227 (246h51m45.530126251s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:38:00.506886217 (246h51m47.326122241s) [upload=237005]: Next returned nil segment Apr 17 22:38:00.506910955 (246h51m47.326146989s) [upload=237005]: Waiting for error group managing segments... Apr 17 22:38:00.795192183 (246h51m47.614428217s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.802966977 (246h51m47.622203012s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.837150420 (246h51m47.656386444s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.857273577 (246h51m47.676509601s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:01.831163825 (246h51m48.650399849s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.129931912 (246h51m48.949167936s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.167981246 (246h51m48.98721728s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.179145783 (246h51m48.998381807s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.275673416 (246h51m49.09490945s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.654733063 (246h51m49.473969097s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.733948735 (246h51m49.553184759s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.806700572 (246h51m49.625936596s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807972947 (246h51m49.627208972s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807993647 (246h51m49.627229681s) [seg_pos={0 0}, upload=237005]: Satisfied cohort requirements after 13 pieces Apr 17 22:38:02.808282966 (246h51m49.62751901s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: expected piece hash; context canceled; EOF Apr 17 22:38:02.808293136 (246h51m49.62752917s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808429940 (246h51m49.627665964s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection Apr 17 22:38:02.808441132 (246h51m49.627677166s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808538750 (246h51m49.627774784s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QtY3yN6J1H8htRneePHu4RmbN7A1xkgywphMDVsVxhTczGupY): protocol: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection Apr 17 22:38:02.808549581 (246h51m49.627785615s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808579067 (246h51m49.627815091s) [seg_pos={0 0}, upload=237005]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 17 22:38:02.808667418 (246h51m49.627903452s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 17 22:38:02.808674110 (246h51m49.627910134s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 17 22:38:03.086673887 (246h51m49.905909911s) [upload=237005]: Done uploading segments... Apr 17 22:38:03.086701991 (246h51m49.905938015s) [upload=237005]: Upload finished. err: <nil> Duration: 7.473002045s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 29 out of 60 is failed (threshold=6) [XXXX...X..X.X...X.X......XXXX.XX..XXXXXXXX.XXXXX......X.....] duration: 4.769775252s trace: https://jaeger.dev.storj.io/trace/139507385cb7d1ef FAIL (2026-04-17 22:57:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.769804078s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 29 out of 60 is failed (threshold=6) [.XXXX...X..X.X...X.X......XXXX.XX..XXXXXXXX.XXXXX......X....] duration: 4.212900007s trace: https://jaeger.dev.storj.io/trace/139507385cb7d1ef FAIL (2026-04-17 22:57:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.212919816s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 10 out of 60 is failed (threshold=3) [.............XXXXXXXXXX.....................................] duration: 9.057657573s trace: https://jaeger.dev.storj.io/trace/51ac1c35b102be61 FAIL (2026-04-17 22:56:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 9.057684765s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.........X.....X...X...X........XX..........................] duration: 8.325135363s trace: https://jaeger.dev.storj.io/trace/1e79bf0cbb46f7f9 FAIL (2026-04-17 22:56:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ments... Apr 17 22:37:58.471270102 (246h51m45.290506126s) [upload=237005]: Got next segment. Inline: false Apr 17 22:37:58.471378241 (246h51m45.290614265s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 17 22:37:58.471387018 (246h51m45.290623052s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 17 22:37:58.708193142 (246h51m45.527429166s) [seg_pos={0 0}, upload=237005]: Begin upload segment... Apr 17 22:37:58.708247858 (246h51m45.527483882s) [seg_pos={0 0}, upload=237005]: Done begin upload segment. Apr 17 22:37:58.708272956 (246h51m45.52750898s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708361497 (246h51m45.527597561s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708820833 (246h51m45.528056857s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708977306 (246h51m45.52821333s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709119320 (246h51m45.528355344s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709308925 (246h51m45.52854495s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709392147 (246h51m45.528628171s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709605307 (246h51m45.528841331s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709666606 (246h51m45.52890263s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709922579 (246h51m45.529158603s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709971264 (246h51m45.529207288s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710303314 (246h51m45.529539338s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710310148 (246h51m45.529546172s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710527357 (246h51m45.529763381s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710597883 (246h51m45.529833907s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710890227 (246h51m45.530126251s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:38:00.506886217 (246h51m47.326122241s) [upload=237005]: Next returned nil segment Apr 17 22:38:00.506910955 (246h51m47.326146989s) [upload=237005]: Waiting for error group managing segments... Apr 17 22:38:00.795192183 (246h51m47.614428217s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.802966977 (246h51m47.622203012s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.837150420 (246h51m47.656386444s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.857273577 (246h51m47.676509601s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:01.831163825 (246h51m48.650399849s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.129931912 (246h51m48.949167936s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.167981246 (246h51m48.98721728s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.179145783 (246h51m48.998381807s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.275673416 (246h51m49.09490945s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.654733063 (246h51m49.473969097s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.733948735 (246h51m49.553184759s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.806700572 (246h51m49.625936596s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807972947 (246h51m49.627208972s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807993647 (246h51m49.627229681s) [seg_pos={0 0}, upload=237005]: Satisfied cohort requirements after 13 pieces Apr 17 22:38:02.808282966 (246h51m49.62751901s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: expected piece hash; context canceled; EOF Apr 17 22:38:02.808293136 (246h51m49.62752917s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808429940 (246h51m49.627665964s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection Apr 17 22:38:02.808441132 (246h51m49.627677166s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808538750 (246h51m49.627774784s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QtY3yN6J1H8htRneePHu4RmbN7A1xkgywphMDVsVxhTczGupY): protocol: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection Apr 17 22:38:02.808549581 (246h51m49.627785615s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808579067 (246h51m49.627815091s) [seg_pos={0 0}, upload=237005]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 17 22:38:02.808667418 (246h51m49.627903452s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 17 22:38:02.808674110 (246h51m49.627910134s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 17 22:38:03.086673887 (246h51m49.905909911s) [upload=237005]: Done uploading segments... Apr 17 22:38:03.086701991 (246h51m49.905938015s) [upload=237005]: Upload finished. err: <nil> Duration: 8.325171714s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 10 out of 60 is failed (threshold=3) [..............XXXXXXXXXX....................................] duration: 9.923031008s trace: https://jaeger.dev.storj.io/trace/51ac1c35b102be61 FAIL (2026-04-17 22:56:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 9.923058271s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 30 out of 60 is failed (threshold=6) [X.XXXX...X..X.X...X.X......XXXX.XX..XXXXXXXX.XXXXX......X...] duration: 4.074014573s trace: https://jaeger.dev.storj.io/trace/139507385cb7d1ef FAIL (2026-04-17 22:56:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.074046084s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 11 out of 60 is failed (threshold=3) [X..............XXXXXXXXXX...................................] duration: 10.819334467s trace: https://jaeger.dev.storj.io/trace/51ac1c35b102be61 FAIL (2026-04-17 22:56:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 10.819369036s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 31 out of 60 is failed (threshold=6) [XX.XXXX...X..X.X...X.X......XXXX.XX..XXXXXXXX.XXXXX......X..] duration: 4.906323869s trace: https://jaeger.dev.storj.io/trace/139507385cb7d1ef FAIL (2026-04-17 22:55:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.906351994s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..........X.....X...X...X........XX.........................] duration: 7.345586752s trace: https://jaeger.dev.storj.io/trace/1e79bf0cbb46f7f9 FAIL (2026-04-17 22:55:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ments... Apr 17 22:37:58.471270102 (246h51m45.290506126s) [upload=237005]: Got next segment. Inline: false Apr 17 22:37:58.471378241 (246h51m45.290614265s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 17 22:37:58.471387018 (246h51m45.290623052s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 17 22:37:58.708193142 (246h51m45.527429166s) [seg_pos={0 0}, upload=237005]: Begin upload segment... Apr 17 22:37:58.708247858 (246h51m45.527483882s) [seg_pos={0 0}, upload=237005]: Done begin upload segment. Apr 17 22:37:58.708272956 (246h51m45.52750898s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708361497 (246h51m45.527597561s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708820833 (246h51m45.528056857s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708977306 (246h51m45.52821333s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709119320 (246h51m45.528355344s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709308925 (246h51m45.52854495s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709392147 (246h51m45.528628171s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709605307 (246h51m45.528841331s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709666606 (246h51m45.52890263s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709922579 (246h51m45.529158603s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709971264 (246h51m45.529207288s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710303314 (246h51m45.529539338s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710310148 (246h51m45.529546172s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710527357 (246h51m45.529763381s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710597883 (246h51m45.529833907s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710890227 (246h51m45.530126251s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:38:00.506886217 (246h51m47.326122241s) [upload=237005]: Next returned nil segment Apr 17 22:38:00.506910955 (246h51m47.326146989s) [upload=237005]: Waiting for error group managing segments... Apr 17 22:38:00.795192183 (246h51m47.614428217s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.802966977 (246h51m47.622203012s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.837150420 (246h51m47.656386444s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.857273577 (246h51m47.676509601s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:01.831163825 (246h51m48.650399849s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.129931912 (246h51m48.949167936s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.167981246 (246h51m48.98721728s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.179145783 (246h51m48.998381807s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.275673416 (246h51m49.09490945s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.654733063 (246h51m49.473969097s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.733948735 (246h51m49.553184759s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.806700572 (246h51m49.625936596s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807972947 (246h51m49.627208972s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807993647 (246h51m49.627229681s) [seg_pos={0 0}, upload=237005]: Satisfied cohort requirements after 13 pieces Apr 17 22:38:02.808282966 (246h51m49.62751901s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: expected piece hash; context canceled; EOF Apr 17 22:38:02.808293136 (246h51m49.62752917s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808429940 (246h51m49.627665964s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection Apr 17 22:38:02.808441132 (246h51m49.627677166s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808538750 (246h51m49.627774784s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QtY3yN6J1H8htRneePHu4RmbN7A1xkgywphMDVsVxhTczGupY): protocol: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection Apr 17 22:38:02.808549581 (246h51m49.627785615s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808579067 (246h51m49.627815091s) [seg_pos={0 0}, upload=237005]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 17 22:38:02.808667418 (246h51m49.627903452s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 17 22:38:02.808674110 (246h51m49.627910134s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 17 22:38:03.086673887 (246h51m49.905909911s) [upload=237005]: Done uploading segments... Apr 17 22:38:03.086701991 (246h51m49.905938015s) [upload=237005]: Upload finished. err: <nil> Duration: 7.345614486s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [X..........X.....X...X...X........XX........................] duration: 7.320152317s trace: https://jaeger.dev.storj.io/trace/1e79bf0cbb46f7f9 FAIL (2026-04-17 22:55:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ments... Apr 17 22:37:58.471270102 (246h51m45.290506126s) [upload=237005]: Got next segment. Inline: false Apr 17 22:37:58.471378241 (246h51m45.290614265s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 17 22:37:58.471387018 (246h51m45.290623052s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 17 22:37:58.708193142 (246h51m45.527429166s) [seg_pos={0 0}, upload=237005]: Begin upload segment... Apr 17 22:37:58.708247858 (246h51m45.527483882s) [seg_pos={0 0}, upload=237005]: Done begin upload segment. Apr 17 22:37:58.708272956 (246h51m45.52750898s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708361497 (246h51m45.527597561s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708820833 (246h51m45.528056857s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708977306 (246h51m45.52821333s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709119320 (246h51m45.528355344s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709308925 (246h51m45.52854495s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709392147 (246h51m45.528628171s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709605307 (246h51m45.528841331s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709666606 (246h51m45.52890263s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709922579 (246h51m45.529158603s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709971264 (246h51m45.529207288s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710303314 (246h51m45.529539338s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710310148 (246h51m45.529546172s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710527357 (246h51m45.529763381s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710597883 (246h51m45.529833907s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710890227 (246h51m45.530126251s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:38:00.506886217 (246h51m47.326122241s) [upload=237005]: Next returned nil segment Apr 17 22:38:00.506910955 (246h51m47.326146989s) [upload=237005]: Waiting for error group managing segments... Apr 17 22:38:00.795192183 (246h51m47.614428217s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.802966977 (246h51m47.622203012s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.837150420 (246h51m47.656386444s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.857273577 (246h51m47.676509601s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:01.831163825 (246h51m48.650399849s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.129931912 (246h51m48.949167936s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.167981246 (246h51m48.98721728s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.179145783 (246h51m48.998381807s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.275673416 (246h51m49.09490945s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.654733063 (246h51m49.473969097s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.733948735 (246h51m49.553184759s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.806700572 (246h51m49.625936596s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807972947 (246h51m49.627208972s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807993647 (246h51m49.627229681s) [seg_pos={0 0}, upload=237005]: Satisfied cohort requirements after 13 pieces Apr 17 22:38:02.808282966 (246h51m49.62751901s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: expected piece hash; context canceled; EOF Apr 17 22:38:02.808293136 (246h51m49.62752917s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808429940 (246h51m49.627665964s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection Apr 17 22:38:02.808441132 (246h51m49.627677166s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808538750 (246h51m49.627774784s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QtY3yN6J1H8htRneePHu4RmbN7A1xkgywphMDVsVxhTczGupY): protocol: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection Apr 17 22:38:02.808549581 (246h51m49.627785615s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808579067 (246h51m49.627815091s) [seg_pos={0 0}, upload=237005]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 17 22:38:02.808667418 (246h51m49.627903452s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 17 22:38:02.808674110 (246h51m49.627910134s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 17 22:38:03.086673887 (246h51m49.905909911s) [upload=237005]: Done uploading segments... Apr 17 22:38:03.086701991 (246h51m49.905938015s) [upload=237005]: Upload finished. err: <nil> Duration: 7.320181704s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 12 out of 60 is failed (threshold=3) [XX..............XXXXXXXXXX..................................] duration: 10.196679533s trace: https://jaeger.dev.storj.io/trace/51ac1c35b102be61 FAIL (2026-04-17 22:54:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 10.196705563s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 31 out of 60 is failed (threshold=6) [.XX.XXXX...X..X.X...X.X......XXXX.XX..XXXXXXXX.XXXXX......X.] duration: 4.996541734s trace: https://jaeger.dev.storj.io/trace/139507385cb7d1ef FAIL (2026-04-17 22:54:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.996593192s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 13 out of 60 is failed (threshold=3) [XXX..............XXXXXXXXXX.................................] duration: 10.159326385s trace: https://jaeger.dev.storj.io/trace/51ac1c35b102be61 FAIL (2026-04-17 22:54:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 10.159366984s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.X..........X.....X...X...X........XX.......................] duration: 7.356910276s trace: https://jaeger.dev.storj.io/trace/1e79bf0cbb46f7f9 FAIL (2026-04-17 22:54:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ments... Apr 17 22:37:58.471270102 (246h51m45.290506126s) [upload=237005]: Got next segment. Inline: false Apr 17 22:37:58.471378241 (246h51m45.290614265s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 17 22:37:58.471387018 (246h51m45.290623052s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 17 22:37:58.708193142 (246h51m45.527429166s) [seg_pos={0 0}, upload=237005]: Begin upload segment... Apr 17 22:37:58.708247858 (246h51m45.527483882s) [seg_pos={0 0}, upload=237005]: Done begin upload segment. Apr 17 22:37:58.708272956 (246h51m45.52750898s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708361497 (246h51m45.527597561s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708820833 (246h51m45.528056857s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708977306 (246h51m45.52821333s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709119320 (246h51m45.528355344s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709308925 (246h51m45.52854495s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709392147 (246h51m45.528628171s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709605307 (246h51m45.528841331s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709666606 (246h51m45.52890263s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709922579 (246h51m45.529158603s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709971264 (246h51m45.529207288s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710303314 (246h51m45.529539338s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710310148 (246h51m45.529546172s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710527357 (246h51m45.529763381s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710597883 (246h51m45.529833907s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710890227 (246h51m45.530126251s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:38:00.506886217 (246h51m47.326122241s) [upload=237005]: Next returned nil segment Apr 17 22:38:00.506910955 (246h51m47.326146989s) [upload=237005]: Waiting for error group managing segments... Apr 17 22:38:00.795192183 (246h51m47.614428217s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.802966977 (246h51m47.622203012s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.837150420 (246h51m47.656386444s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.857273577 (246h51m47.676509601s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:01.831163825 (246h51m48.650399849s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.129931912 (246h51m48.949167936s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.167981246 (246h51m48.98721728s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.179145783 (246h51m48.998381807s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.275673416 (246h51m49.09490945s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.654733063 (246h51m49.473969097s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.733948735 (246h51m49.553184759s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.806700572 (246h51m49.625936596s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807972947 (246h51m49.627208972s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807993647 (246h51m49.627229681s) [seg_pos={0 0}, upload=237005]: Satisfied cohort requirements after 13 pieces Apr 17 22:38:02.808282966 (246h51m49.62751901s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: expected piece hash; context canceled; EOF Apr 17 22:38:02.808293136 (246h51m49.62752917s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808429940 (246h51m49.627665964s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection Apr 17 22:38:02.808441132 (246h51m49.627677166s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808538750 (246h51m49.627774784s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QtY3yN6J1H8htRneePHu4RmbN7A1xkgywphMDVsVxhTczGupY): protocol: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection Apr 17 22:38:02.808549581 (246h51m49.627785615s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808579067 (246h51m49.627815091s) [seg_pos={0 0}, upload=237005]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 17 22:38:02.808667418 (246h51m49.627903452s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 17 22:38:02.808674110 (246h51m49.627910134s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 17 22:38:03.086673887 (246h51m49.905909911s) [upload=237005]: Done uploading segments... Apr 17 22:38:03.086701991 (246h51m49.905938015s) [upload=237005]: Upload finished. err: <nil> Duration: 7.356940655s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 32 out of 60 is failed (threshold=6) [X.XX.XXXX...X..X.X...X.X......XXXX.XX..XXXXXXXX.XXXXX......X] duration: 5.000689226s trace: https://jaeger.dev.storj.io/trace/139507385cb7d1ef FAIL (2026-04-17 22:53:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000708072s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [..X..........X.....X...X...X........XX......................] duration: 7.361846102s trace: https://jaeger.dev.storj.io/trace/1e79bf0cbb46f7f9 FAIL (2026-04-17 22:53:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ments... Apr 17 22:37:58.471270102 (246h51m45.290506126s) [upload=237005]: Got next segment. Inline: false Apr 17 22:37:58.471378241 (246h51m45.290614265s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 17 22:37:58.471387018 (246h51m45.290623052s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 17 22:37:58.708193142 (246h51m45.527429166s) [seg_pos={0 0}, upload=237005]: Begin upload segment... Apr 17 22:37:58.708247858 (246h51m45.527483882s) [seg_pos={0 0}, upload=237005]: Done begin upload segment. Apr 17 22:37:58.708272956 (246h51m45.52750898s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708361497 (246h51m45.527597561s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708820833 (246h51m45.528056857s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708977306 (246h51m45.52821333s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709119320 (246h51m45.528355344s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709308925 (246h51m45.52854495s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709392147 (246h51m45.528628171s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709605307 (246h51m45.528841331s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709666606 (246h51m45.52890263s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709922579 (246h51m45.529158603s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709971264 (246h51m45.529207288s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710303314 (246h51m45.529539338s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710310148 (246h51m45.529546172s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710527357 (246h51m45.529763381s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710597883 (246h51m45.529833907s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710890227 (246h51m45.530126251s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:38:00.506886217 (246h51m47.326122241s) [upload=237005]: Next returned nil segment Apr 17 22:38:00.506910955 (246h51m47.326146989s) [upload=237005]: Waiting for error group managing segments... Apr 17 22:38:00.795192183 (246h51m47.614428217s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.802966977 (246h51m47.622203012s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.837150420 (246h51m47.656386444s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.857273577 (246h51m47.676509601s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:01.831163825 (246h51m48.650399849s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.129931912 (246h51m48.949167936s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.167981246 (246h51m48.98721728s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.179145783 (246h51m48.998381807s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.275673416 (246h51m49.09490945s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.654733063 (246h51m49.473969097s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.733948735 (246h51m49.553184759s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.806700572 (246h51m49.625936596s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807972947 (246h51m49.627208972s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807993647 (246h51m49.627229681s) [seg_pos={0 0}, upload=237005]: Satisfied cohort requirements after 13 pieces Apr 17 22:38:02.808282966 (246h51m49.62751901s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: expected piece hash; context canceled; EOF Apr 17 22:38:02.808293136 (246h51m49.62752917s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808429940 (246h51m49.627665964s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection Apr 17 22:38:02.808441132 (246h51m49.627677166s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808538750 (246h51m49.627774784s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QtY3yN6J1H8htRneePHu4RmbN7A1xkgywphMDVsVxhTczGupY): protocol: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection Apr 17 22:38:02.808549581 (246h51m49.627785615s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808579067 (246h51m49.627815091s) [seg_pos={0 0}, upload=237005]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 17 22:38:02.808667418 (246h51m49.627903452s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 17 22:38:02.808674110 (246h51m49.627910134s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 17 22:38:03.086673887 (246h51m49.905909911s) [upload=237005]: Done uploading segments... Apr 17 22:38:03.086701991 (246h51m49.905938015s) [upload=237005]: Upload finished. err: <nil> Duration: 7.361873004s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 14 out of 60 is failed (threshold=3) [XXXX..............XXXXXXXXXX................................] duration: 8.691025065s trace: https://jaeger.dev.storj.io/trace/51ac1c35b102be61 FAIL (2026-04-17 22:52:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.69107929s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 32 out of 60 is failed (threshold=6) [XX.XX.XXXX...X..X.X...X.X......XXXX.XX..XXXXXXXX.XXXXX......] duration: 5.000394802s trace: https://jaeger.dev.storj.io/trace/3bb80521ccaf93c8 FAIL (2026-04-17 22:52:47 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000416253s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [...X..........X.....X...X...X........XX.....................] duration: 15.764053988s trace: https://jaeger.dev.storj.io/trace/1e79bf0cbb46f7f9 FAIL (2026-04-17 22:52:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ments... Apr 17 22:37:58.471270102 (246h51m45.290506126s) [upload=237005]: Got next segment. Inline: false Apr 17 22:37:58.471378241 (246h51m45.290614265s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 17 22:37:58.471387018 (246h51m45.290623052s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 17 22:37:58.708193142 (246h51m45.527429166s) [seg_pos={0 0}, upload=237005]: Begin upload segment... Apr 17 22:37:58.708247858 (246h51m45.527483882s) [seg_pos={0 0}, upload=237005]: Done begin upload segment. Apr 17 22:37:58.708272956 (246h51m45.52750898s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708361497 (246h51m45.527597561s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708820833 (246h51m45.528056857s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708977306 (246h51m45.52821333s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709119320 (246h51m45.528355344s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709308925 (246h51m45.52854495s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709392147 (246h51m45.528628171s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709605307 (246h51m45.528841331s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709666606 (246h51m45.52890263s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709922579 (246h51m45.529158603s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709971264 (246h51m45.529207288s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710303314 (246h51m45.529539338s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710310148 (246h51m45.529546172s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710527357 (246h51m45.529763381s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710597883 (246h51m45.529833907s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710890227 (246h51m45.530126251s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:38:00.506886217 (246h51m47.326122241s) [upload=237005]: Next returned nil segment Apr 17 22:38:00.506910955 (246h51m47.326146989s) [upload=237005]: Waiting for error group managing segments... Apr 17 22:38:00.795192183 (246h51m47.614428217s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.802966977 (246h51m47.622203012s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.837150420 (246h51m47.656386444s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.857273577 (246h51m47.676509601s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:01.831163825 (246h51m48.650399849s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.129931912 (246h51m48.949167936s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.167981246 (246h51m48.98721728s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.179145783 (246h51m48.998381807s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.275673416 (246h51m49.09490945s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.654733063 (246h51m49.473969097s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.733948735 (246h51m49.553184759s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.806700572 (246h51m49.625936596s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807972947 (246h51m49.627208972s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807993647 (246h51m49.627229681s) [seg_pos={0 0}, upload=237005]: Satisfied cohort requirements after 13 pieces Apr 17 22:38:02.808282966 (246h51m49.62751901s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: expected piece hash; context canceled; EOF Apr 17 22:38:02.808293136 (246h51m49.62752917s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808429940 (246h51m49.627665964s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection Apr 17 22:38:02.808441132 (246h51m49.627677166s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808538750 (246h51m49.627774784s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QtY3yN6J1H8htRneePHu4RmbN7A1xkgywphMDVsVxhTczGupY): protocol: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection Apr 17 22:38:02.808549581 (246h51m49.627785615s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808579067 (246h51m49.627815091s) [seg_pos={0 0}, upload=237005]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 17 22:38:02.808667418 (246h51m49.627903452s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 17 22:38:02.808674110 (246h51m49.627910134s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 17 22:38:03.086673887 (246h51m49.905909911s) [upload=237005]: Done uploading segments... Apr 17 22:38:03.086701991 (246h51m49.905938015s) [upload=237005]: Upload finished. err: <nil> Duration: 15.764082443s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 15 out of 60 is failed (threshold=3) [XXXXX..............XXXXXXXXXX...............................] duration: 8.728410245s trace: https://jaeger.dev.storj.io/trace/51ac1c35b102be61 FAIL (2026-04-17 22:52:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.728437126s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [....X..........X.....X...X...X........XX....................] duration: 8.757691146s trace: https://jaeger.dev.storj.io/trace/1e79bf0cbb46f7f9 FAIL (2026-04-17 22:51:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ments... Apr 17 22:37:58.471270102 (246h51m45.290506126s) [upload=237005]: Got next segment. Inline: false Apr 17 22:37:58.471378241 (246h51m45.290614265s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 17 22:37:58.471387018 (246h51m45.290623052s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 17 22:37:58.708193142 (246h51m45.527429166s) [seg_pos={0 0}, upload=237005]: Begin upload segment... Apr 17 22:37:58.708247858 (246h51m45.527483882s) [seg_pos={0 0}, upload=237005]: Done begin upload segment. Apr 17 22:37:58.708272956 (246h51m45.52750898s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708361497 (246h51m45.527597561s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708820833 (246h51m45.528056857s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708977306 (246h51m45.52821333s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709119320 (246h51m45.528355344s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709308925 (246h51m45.52854495s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709392147 (246h51m45.528628171s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709605307 (246h51m45.528841331s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709666606 (246h51m45.52890263s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709922579 (246h51m45.529158603s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709971264 (246h51m45.529207288s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710303314 (246h51m45.529539338s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710310148 (246h51m45.529546172s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710527357 (246h51m45.529763381s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710597883 (246h51m45.529833907s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710890227 (246h51m45.530126251s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:38:00.506886217 (246h51m47.326122241s) [upload=237005]: Next returned nil segment Apr 17 22:38:00.506910955 (246h51m47.326146989s) [upload=237005]: Waiting for error group managing segments... Apr 17 22:38:00.795192183 (246h51m47.614428217s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.802966977 (246h51m47.622203012s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.837150420 (246h51m47.656386444s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.857273577 (246h51m47.676509601s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:01.831163825 (246h51m48.650399849s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.129931912 (246h51m48.949167936s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.167981246 (246h51m48.98721728s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.179145783 (246h51m48.998381807s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.275673416 (246h51m49.09490945s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.654733063 (246h51m49.473969097s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.733948735 (246h51m49.553184759s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.806700572 (246h51m49.625936596s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807972947 (246h51m49.627208972s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807993647 (246h51m49.627229681s) [seg_pos={0 0}, upload=237005]: Satisfied cohort requirements after 13 pieces Apr 17 22:38:02.808282966 (246h51m49.62751901s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: expected piece hash; context canceled; EOF Apr 17 22:38:02.808293136 (246h51m49.62752917s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808429940 (246h51m49.627665964s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection Apr 17 22:38:02.808441132 (246h51m49.627677166s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808538750 (246h51m49.627774784s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QtY3yN6J1H8htRneePHu4RmbN7A1xkgywphMDVsVxhTczGupY): protocol: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection Apr 17 22:38:02.808549581 (246h51m49.627785615s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808579067 (246h51m49.627815091s) [seg_pos={0 0}, upload=237005]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 17 22:38:02.808667418 (246h51m49.627903452s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 17 22:38:02.808674110 (246h51m49.627910134s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 17 22:38:03.086673887 (246h51m49.905909911s) [upload=237005]: Done uploading segments... Apr 17 22:38:03.086701991 (246h51m49.905938015s) [upload=237005]: Upload finished. err: <nil> Duration: 8.757733989s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 32 out of 60 is failed (threshold=6) [.XX.XX.XXXX...X..X.X...X.X......XXXX.XX..XXXXXXXX.XXXXX.....] duration: 4.208470554s trace: https://jaeger.dev.storj.io/trace/3bb80521ccaf93c8 FAIL (2026-04-17 22:51:53 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.208494672s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 33 out of 60 is failed (threshold=6) [X.XX.XX.XXXX...X..X.X...X.X......XXXX.XX..XXXXXXXX.XXXXX....] duration: 3.881694588s trace: https://jaeger.dev.storj.io/trace/3bb80521ccaf93c8 FAIL (2026-04-17 22:51:35 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.881719637s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.....X..........X.....X...X...X........XX...................] duration: 7.026470178s trace: https://jaeger.dev.storj.io/trace/1e79bf0cbb46f7f9 FAIL (2026-04-17 22:51:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ments... Apr 17 22:37:58.471270102 (246h51m45.290506126s) [upload=237005]: Got next segment. Inline: false Apr 17 22:37:58.471378241 (246h51m45.290614265s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 17 22:37:58.471387018 (246h51m45.290623052s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 17 22:37:58.708193142 (246h51m45.527429166s) [seg_pos={0 0}, upload=237005]: Begin upload segment... Apr 17 22:37:58.708247858 (246h51m45.527483882s) [seg_pos={0 0}, upload=237005]: Done begin upload segment. Apr 17 22:37:58.708272956 (246h51m45.52750898s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708361497 (246h51m45.527597561s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708820833 (246h51m45.528056857s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708977306 (246h51m45.52821333s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709119320 (246h51m45.528355344s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709308925 (246h51m45.52854495s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709392147 (246h51m45.528628171s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709605307 (246h51m45.528841331s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709666606 (246h51m45.52890263s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709922579 (246h51m45.529158603s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709971264 (246h51m45.529207288s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710303314 (246h51m45.529539338s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710310148 (246h51m45.529546172s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710527357 (246h51m45.529763381s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710597883 (246h51m45.529833907s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710890227 (246h51m45.530126251s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:38:00.506886217 (246h51m47.326122241s) [upload=237005]: Next returned nil segment Apr 17 22:38:00.506910955 (246h51m47.326146989s) [upload=237005]: Waiting for error group managing segments... Apr 17 22:38:00.795192183 (246h51m47.614428217s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.802966977 (246h51m47.622203012s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.837150420 (246h51m47.656386444s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.857273577 (246h51m47.676509601s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:01.831163825 (246h51m48.650399849s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.129931912 (246h51m48.949167936s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.167981246 (246h51m48.98721728s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.179145783 (246h51m48.998381807s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.275673416 (246h51m49.09490945s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.654733063 (246h51m49.473969097s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.733948735 (246h51m49.553184759s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.806700572 (246h51m49.625936596s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807972947 (246h51m49.627208972s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807993647 (246h51m49.627229681s) [seg_pos={0 0}, upload=237005]: Satisfied cohort requirements after 13 pieces Apr 17 22:38:02.808282966 (246h51m49.62751901s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: expected piece hash; context canceled; EOF Apr 17 22:38:02.808293136 (246h51m49.62752917s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808429940 (246h51m49.627665964s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection Apr 17 22:38:02.808441132 (246h51m49.627677166s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808538750 (246h51m49.627774784s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QtY3yN6J1H8htRneePHu4RmbN7A1xkgywphMDVsVxhTczGupY): protocol: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection Apr 17 22:38:02.808549581 (246h51m49.627785615s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808579067 (246h51m49.627815091s) [seg_pos={0 0}, upload=237005]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 17 22:38:02.808667418 (246h51m49.627903452s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 17 22:38:02.808674110 (246h51m49.627910134s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 17 22:38:03.086673887 (246h51m49.905909911s) [upload=237005]: Done uploading segments... Apr 17 22:38:03.086701991 (246h51m49.905938015s) [upload=237005]: Upload finished. err: <nil> Duration: 7.026495527s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 33 out of 60 is failed (threshold=6) [.X.XX.XX.XXXX...X..X.X...X.X......XXXX.XX..XXXXXXXX.XXXXX...] duration: 3.996346709s trace: https://jaeger.dev.storj.io/trace/3bb80521ccaf93c8 FAIL (2026-04-17 22:51:15 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.996370236s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [......X..........X.....X...X...X........XX..................] duration: 7.844474238s trace: https://jaeger.dev.storj.io/trace/1e79bf0cbb46f7f9 FAIL (2026-04-17 22:51:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ments... Apr 17 22:37:58.471270102 (246h51m45.290506126s) [upload=237005]: Got next segment. Inline: false Apr 17 22:37:58.471378241 (246h51m45.290614265s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 17 22:37:58.471387018 (246h51m45.290623052s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 17 22:37:58.708193142 (246h51m45.527429166s) [seg_pos={0 0}, upload=237005]: Begin upload segment... Apr 17 22:37:58.708247858 (246h51m45.527483882s) [seg_pos={0 0}, upload=237005]: Done begin upload segment. Apr 17 22:37:58.708272956 (246h51m45.52750898s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708361497 (246h51m45.527597561s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708820833 (246h51m45.528056857s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708977306 (246h51m45.52821333s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709119320 (246h51m45.528355344s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709308925 (246h51m45.52854495s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709392147 (246h51m45.528628171s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709605307 (246h51m45.528841331s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709666606 (246h51m45.52890263s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709922579 (246h51m45.529158603s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709971264 (246h51m45.529207288s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710303314 (246h51m45.529539338s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710310148 (246h51m45.529546172s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710527357 (246h51m45.529763381s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710597883 (246h51m45.529833907s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710890227 (246h51m45.530126251s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:38:00.506886217 (246h51m47.326122241s) [upload=237005]: Next returned nil segment Apr 17 22:38:00.506910955 (246h51m47.326146989s) [upload=237005]: Waiting for error group managing segments... Apr 17 22:38:00.795192183 (246h51m47.614428217s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.802966977 (246h51m47.622203012s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.837150420 (246h51m47.656386444s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.857273577 (246h51m47.676509601s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:01.831163825 (246h51m48.650399849s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.129931912 (246h51m48.949167936s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.167981246 (246h51m48.98721728s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.179145783 (246h51m48.998381807s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.275673416 (246h51m49.09490945s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.654733063 (246h51m49.473969097s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.733948735 (246h51m49.553184759s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.806700572 (246h51m49.625936596s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807972947 (246h51m49.627208972s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807993647 (246h51m49.627229681s) [seg_pos={0 0}, upload=237005]: Satisfied cohort requirements after 13 pieces Apr 17 22:38:02.808282966 (246h51m49.62751901s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: expected piece hash; context canceled; EOF Apr 17 22:38:02.808293136 (246h51m49.62752917s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808429940 (246h51m49.627665964s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection Apr 17 22:38:02.808441132 (246h51m49.627677166s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808538750 (246h51m49.627774784s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QtY3yN6J1H8htRneePHu4RmbN7A1xkgywphMDVsVxhTczGupY): protocol: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection Apr 17 22:38:02.808549581 (246h51m49.627785615s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808579067 (246h51m49.627815091s) [seg_pos={0 0}, upload=237005]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 17 22:38:02.808667418 (246h51m49.627903452s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 17 22:38:02.808674110 (246h51m49.627910134s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 17 22:38:03.086673887 (246h51m49.905909911s) [upload=237005]: Done uploading segments... Apr 17 22:38:03.086701991 (246h51m49.905938015s) [upload=237005]: Upload finished. err: <nil> Duration: 7.844504135s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 15 out of 60 is failed (threshold=3) [.XXXXX..............XXXXXXXXXX..............................] duration: 8.448705126s trace: https://jaeger.dev.storj.io/trace/51ac1c35b102be61 FAIL (2026-04-17 22:51:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.448728682s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 34 out of 60 is failed (threshold=6) [X.X.XX.XX.XXXX...X..X.X...X.X......XXXX.XX..XXXXXXXX.XXXXX..] duration: 4.197516865s trace: https://jaeger.dev.storj.io/trace/3bb80521ccaf93c8 FAIL (2026-04-17 22:50:57 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.197533839s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.......X..........X.....X...X...X........XX.................] duration: 8.844210054s trace: https://jaeger.dev.storj.io/trace/1e79bf0cbb46f7f9 FAIL (2026-04-17 22:50:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ments... Apr 17 22:37:58.471270102 (246h51m45.290506126s) [upload=237005]: Got next segment. Inline: false Apr 17 22:37:58.471378241 (246h51m45.290614265s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Apr 17 22:37:58.471387018 (246h51m45.290623052s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentBegin Apr 17 22:37:58.708193142 (246h51m45.527429166s) [seg_pos={0 0}, upload=237005]: Begin upload segment... Apr 17 22:37:58.708247858 (246h51m45.527483882s) [seg_pos={0 0}, upload=237005]: Done begin upload segment. Apr 17 22:37:58.708272956 (246h51m45.52750898s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708361497 (246h51m45.527597561s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708820833 (246h51m45.528056857s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.708977306 (246h51m45.52821333s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709119320 (246h51m45.528355344s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709308925 (246h51m45.52854495s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709392147 (246h51m45.528628171s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709605307 (246h51m45.528841331s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709666606 (246h51m45.52890263s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709922579 (246h51m45.529158603s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.709971264 (246h51m45.529207288s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710303314 (246h51m45.529539338s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710310148 (246h51m45.529546172s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710527357 (246h51m45.529763381s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710597883 (246h51m45.529833907s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:37:58.710890227 (246h51m45.530126251s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Uploading piece... Apr 17 22:38:00.506886217 (246h51m47.326122241s) [upload=237005]: Next returned nil segment Apr 17 22:38:00.506910955 (246h51m47.326146989s) [upload=237005]: Waiting for error group managing segments... Apr 17 22:38:00.795192183 (246h51m47.614428217s) [noise=true , address=77.90.31.14:4666 , piece_id=HDSXNC6ZCVZZDTHDBUZGPFXJQT76Z6FMZROKJ4ZGHD5MUUSFIW3A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.802966977 (246h51m47.622203012s) [noise=true , address=77.90.31.10:4521 , piece_id=2USPTFNSVT4QD3MNVHFPDE5GZW3KXARF3SRW3XLOSJXA6RANY4VA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.837150420 (246h51m47.656386444s) [noise=true , address=77.90.31.15:4700 , piece_id=Q4YKXK7S5UXV3VGQDXC7RLVWAQVEJ3BS2WUY3LGG2KXFU4PTTXFA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:00.857273577 (246h51m47.676509601s) [noise=true , address=77.90.31.13:4630 , piece_id=IZY2WSNNJG5ROEOM6CBH6YD25ZRZIE7XMZMSQO6XQ7H2ESKHJ6UQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:01.831163825 (246h51m48.650399849s) [noise=true , address=45.91.116.13:4188 , piece_id=RTVKZTFBJAI5LWLP3G5ST5AHOWLCJME3MZGTZG4BWU3PE2T2WE5A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.129931912 (246h51m48.949167936s) [noise=true , address=145.79.227.39:4016 , piece_id=CGNLPBWTFLFB5K3ADI5BX75J7ERJECZQPVAF7UTC622WYW3JD23A, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.167981246 (246h51m48.98721728s) [noise=true , address=45.91.116.14:4224 , piece_id=ZSZKRRRCUEV36AXYRUSEN2YHCUMVH4D7N4KVEAWQ5OF4XEWZD3JQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.179145783 (246h51m48.998381807s) [noise=true , address=45.91.116.8:4048 , piece_id=WBZ7RKVOR3RDRAWHOM6RACJGR4ZQMWBOSBAYMZ4DPNNSZTWM7GPA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.275673416 (246h51m49.09490945s) [noise=true , address=145.79.227.52:4449 , piece_id=AC4DG67AWU5GNPBOKPHPPIXVZWXZWQJBSWJTK6RRWSKJ7EJWCQKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.654733063 (246h51m49.473969097s) [noise=true , address=145.79.227.6:4262 , piece_id=BUVADRII3ULSE4HZZHPBAWDV5D2CXIL7WZAJ4DYVZP6QCLHADVDQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.733948735 (246h51m49.553184759s) [noise=true , address=145.79.227.25:4623 , piece_id=ZTR3BUXES2W5NVFPIT4RXIN2WOIHJUWLL6KFYBPWQKTOCKY7RZOQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.806700572 (246h51m49.625936596s) [noise=true , address=145.79.227.21:4473 , piece_id=YEYRKNQ2D2466MWD5TOASDZO6NVLXTQZ7HEHMT4JPBTFQO3Q6V2Q, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807972947 (246h51m49.627208972s) [noise=true , address=145.79.227.45:4233 , piece_id=CCSYLQB2SH2O5BUVROK43GOOD2XSN3BDYT72SWBA4GTKLAU2GDQA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: <nil> Apr 17 22:38:02.807993647 (246h51m49.627229681s) [seg_pos={0 0}, upload=237005]: Satisfied cohort requirements after 13 pieces Apr 17 22:38:02.808282966 (246h51m49.62751901s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: expected piece hash; context canceled; EOF Apr 17 22:38:02.808293136 (246h51m49.62752917s) [noise=true , address=45.91.116.6:4989 , piece_id=L5BSUBS32DI74H3T2I56EM3EPCXI7SJ2KFCDHQOYZ76J5NDP5MBA, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808429940 (246h51m49.627665964s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41300->145.79.227.19:4078: use of closed network connection Apr 17 22:38:02.808441132 (246h51m49.627677166s) [noise=true , address=145.79.227.19:4078 , piece_id=O4QR7JF6XGDT5HWLQGSBUQL3RBT54VSJSIDRJWBLCVRJEXOWEOEQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808538750 (246h51m49.627774784s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QtY3yN6J1H8htRneePHu4RmbN7A1xkgywphMDVsVxhTczGupY): protocol: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44240->145.79.227.5:4225: use of closed network connection Apr 17 22:38:02.808549581 (246h51m49.627785615s) [noise=true , address=145.79.227.5:4225 , piece_id=55MLEQTSPA5UWDPJAWWWV6AZ5GACSPSY7KORDN2YUPVG2FAMQIKQ, seg_pos={0 0}, upload=237005]: Long tail cancellation due to optimal threshold Apr 17 22:38:02.808579067 (246h51m49.627815091s) [seg_pos={0 0}, upload=237005]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Apr 17 22:38:02.808667418 (246h51m49.627903452s) [upload=237005]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 17 22:38:02.808674110 (246h51m49.627910134s) [upload=237005]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 17 22:38:03.086673887 (246h51m49.905909911s) [upload=237005]: Done uploading segments... Apr 17 22:38:03.086701991 (246h51m49.905938015s) [upload=237005]: Upload finished. err: <nil> Duration: 8.844235912s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 15 out of 60 is failed (threshold=3) [..XXXXX..............XXXXXXXXXX.............................] duration: 9.210196442s trace: https://jaeger.dev.storj.io/trace/51ac1c35b102be61 FAIL (2026-04-17 22:50:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 9.21022685s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)