Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.481136633s trace: https://jaeger.dev.storj.io/trace/6648ff2bad076ed9 FAIL (2026-05-10 14:15:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT, Piece ID: OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A) context deadline exceeded; piecestore: (Node ID: 1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, Piece ID: T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA) context deadline exceeded ======LOGS====== 21893s) [upload=225259]: Uploading segments... May 10 14:01:16.224858523 (235h13m26.745682258s) [upload=225259]: Got next segment. Inline: false May 10 14:01:16.224887649 (235h13m26.745711404s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 10 14:01:16.224892628 (235h13m26.745716373s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 10 14:01:16.458648925 (235h13m26.97947267s) [seg_pos={0 0}, upload=225259]: Begin upload segment... May 10 14:01:16.458694282 (235h13m26.979518027s) [seg_pos={0 0}, upload=225259]: Done begin upload segment. May 10 14:01:16.458731002 (235h13m26.979554777s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458759456 (235h13m26.979583202s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458843968 (235h13m26.979667713s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459127068 (235h13m26.979950813s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459464842 (235h13m26.980288587s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459745368 (235h13m26.980569113s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460016295 (235h13m26.98084004s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460259599 (235h13m26.981083344s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460587765 (235h13m26.98141151s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.462023354 (235h13m26.982847109s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.463113816 (235h13m26.983937561s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.464084608 (235h13m26.984908353s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.465045181 (235h13m26.985868926s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.466126985 (235h13m26.98695073s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467130310 (235h13m26.987954055s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467480458 (235h13m26.988304203s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:18.171956868 (235h13m28.692780613s) [upload=225259]: Next returned nil segment May 10 14:01:18.172017063 (235h13m28.692840808s) [upload=225259]: Waiting for error group managing segments... May 10 14:01:18.458161229 (235h13m28.978984994s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.463350482 (235h13m28.984174227s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.465465377 (235h13m28.986289122s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.467489280 (235h13m28.988313025s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.470505204 (235h13m28.991328949s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.476761352 (235h13m28.997585097s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.479218581 (235h13m29.000042326s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.487335090 (235h13m29.008158855s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.498784128 (235h13m29.019607873s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.504268754 (235h13m29.025092499s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:19.367580157 (235h13m29.888403903s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:24.805735031 (235h13m35.326558766s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992630767 (235h13m46.513454532s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992651637 (235h13m46.513475392s) [seg_pos={0 0}, upload=225259]: Satisfied cohort requirements after 13 pieces May 10 14:01:35.993179815 (235h13m46.51400356s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection May 10 14:01:35.993188822 (235h13m46.514012567s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993412679 (235h13m46.514236424s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: read tcp 188.166.225.170:44272->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993422178 (235h13m46.514245933s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993607952 (235h13m46.514431697s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bbyRMajztw9frbd1V1meid4vQU1AC343NtZrg6qe27HUyw6bQ): protocol: read tcp 188.166.225.170:57672->45.91.116.8:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993615226 (235h13m46.514438971s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993648048 (235h13m46.514471793s) [seg_pos={0 0}, upload=225259]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 10 14:01:35.993741707 (235h13m46.514565452s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 10 14:01:35.993749172 (235h13m46.514572917s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 10 14:01:36.289004295 (235h13m46.809828041s) [upload=225259]: Done uploading segments... May 10 14:01:36.289022049 (235h13m46.809845794s) [upload=225259]: Upload finished. err: <nil> Duration: 6.481162722s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 7.032983509s trace: https://jaeger.dev.storj.io/trace/57e77812227844e8 FAIL (2026-05-10 14:15:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.033050708s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.678156818s trace: https://jaeger.dev.storj.io/trace/6648ff2bad076ed9 FAIL (2026-05-10 14:15:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT, Piece ID: OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A) context deadline exceeded; piecestore: (Node ID: 1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, Piece ID: T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA) context deadline exceeded ======LOGS====== 21893s) [upload=225259]: Uploading segments... May 10 14:01:16.224858523 (235h13m26.745682258s) [upload=225259]: Got next segment. Inline: false May 10 14:01:16.224887649 (235h13m26.745711404s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 10 14:01:16.224892628 (235h13m26.745716373s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 10 14:01:16.458648925 (235h13m26.97947267s) [seg_pos={0 0}, upload=225259]: Begin upload segment... May 10 14:01:16.458694282 (235h13m26.979518027s) [seg_pos={0 0}, upload=225259]: Done begin upload segment. May 10 14:01:16.458731002 (235h13m26.979554777s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458759456 (235h13m26.979583202s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458843968 (235h13m26.979667713s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459127068 (235h13m26.979950813s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459464842 (235h13m26.980288587s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459745368 (235h13m26.980569113s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460016295 (235h13m26.98084004s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460259599 (235h13m26.981083344s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460587765 (235h13m26.98141151s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.462023354 (235h13m26.982847109s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.463113816 (235h13m26.983937561s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.464084608 (235h13m26.984908353s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.465045181 (235h13m26.985868926s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.466126985 (235h13m26.98695073s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467130310 (235h13m26.987954055s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467480458 (235h13m26.988304203s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:18.171956868 (235h13m28.692780613s) [upload=225259]: Next returned nil segment May 10 14:01:18.172017063 (235h13m28.692840808s) [upload=225259]: Waiting for error group managing segments... May 10 14:01:18.458161229 (235h13m28.978984994s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.463350482 (235h13m28.984174227s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.465465377 (235h13m28.986289122s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.467489280 (235h13m28.988313025s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.470505204 (235h13m28.991328949s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.476761352 (235h13m28.997585097s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.479218581 (235h13m29.000042326s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.487335090 (235h13m29.008158855s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.498784128 (235h13m29.019607873s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.504268754 (235h13m29.025092499s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:19.367580157 (235h13m29.888403903s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:24.805735031 (235h13m35.326558766s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992630767 (235h13m46.513454532s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992651637 (235h13m46.513475392s) [seg_pos={0 0}, upload=225259]: Satisfied cohort requirements after 13 pieces May 10 14:01:35.993179815 (235h13m46.51400356s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection May 10 14:01:35.993188822 (235h13m46.514012567s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993412679 (235h13m46.514236424s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: read tcp 188.166.225.170:44272->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993422178 (235h13m46.514245933s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993607952 (235h13m46.514431697s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bbyRMajztw9frbd1V1meid4vQU1AC343NtZrg6qe27HUyw6bQ): protocol: read tcp 188.166.225.170:57672->45.91.116.8:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993615226 (235h13m46.514438971s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993648048 (235h13m46.514471793s) [seg_pos={0 0}, upload=225259]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 10 14:01:35.993741707 (235h13m46.514565452s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 10 14:01:35.993749172 (235h13m46.514572917s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 10 14:01:36.289004295 (235h13m46.809828041s) [upload=225259]: Done uploading segments... May 10 14:01:36.289022049 (235h13m46.809845794s) [upload=225259]: Upload finished. err: <nil> Duration: 5.678182968s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [................................X.X............X.....X......] duration: 4.253325285s trace: https://jaeger.dev.storj.io/trace/35caab641616e4aa FAIL (2026-05-10 14:15:30 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.2533443s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 6.79302358s trace: https://jaeger.dev.storj.io/trace/57e77812227844e8 FAIL (2026-05-10 14:15:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.793216268s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 6.957359023s trace: https://jaeger.dev.storj.io/trace/57e77812227844e8 FAIL (2026-05-10 14:14:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.957384531s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.806190936s trace: https://jaeger.dev.storj.io/trace/6648ff2bad076ed9 FAIL (2026-05-10 14:14:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT, Piece ID: OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A) context deadline exceeded; piecestore: (Node ID: 1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, Piece ID: T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA) context deadline exceeded ======LOGS====== 21893s) [upload=225259]: Uploading segments... May 10 14:01:16.224858523 (235h13m26.745682258s) [upload=225259]: Got next segment. Inline: false May 10 14:01:16.224887649 (235h13m26.745711404s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 10 14:01:16.224892628 (235h13m26.745716373s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 10 14:01:16.458648925 (235h13m26.97947267s) [seg_pos={0 0}, upload=225259]: Begin upload segment... May 10 14:01:16.458694282 (235h13m26.979518027s) [seg_pos={0 0}, upload=225259]: Done begin upload segment. May 10 14:01:16.458731002 (235h13m26.979554777s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458759456 (235h13m26.979583202s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458843968 (235h13m26.979667713s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459127068 (235h13m26.979950813s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459464842 (235h13m26.980288587s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459745368 (235h13m26.980569113s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460016295 (235h13m26.98084004s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460259599 (235h13m26.981083344s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460587765 (235h13m26.98141151s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.462023354 (235h13m26.982847109s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.463113816 (235h13m26.983937561s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.464084608 (235h13m26.984908353s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.465045181 (235h13m26.985868926s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.466126985 (235h13m26.98695073s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467130310 (235h13m26.987954055s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467480458 (235h13m26.988304203s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:18.171956868 (235h13m28.692780613s) [upload=225259]: Next returned nil segment May 10 14:01:18.172017063 (235h13m28.692840808s) [upload=225259]: Waiting for error group managing segments... May 10 14:01:18.458161229 (235h13m28.978984994s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.463350482 (235h13m28.984174227s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.465465377 (235h13m28.986289122s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.467489280 (235h13m28.988313025s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.470505204 (235h13m28.991328949s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.476761352 (235h13m28.997585097s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.479218581 (235h13m29.000042326s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.487335090 (235h13m29.008158855s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.498784128 (235h13m29.019607873s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.504268754 (235h13m29.025092499s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:19.367580157 (235h13m29.888403903s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:24.805735031 (235h13m35.326558766s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992630767 (235h13m46.513454532s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992651637 (235h13m46.513475392s) [seg_pos={0 0}, upload=225259]: Satisfied cohort requirements after 13 pieces May 10 14:01:35.993179815 (235h13m46.51400356s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection May 10 14:01:35.993188822 (235h13m46.514012567s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993412679 (235h13m46.514236424s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: read tcp 188.166.225.170:44272->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993422178 (235h13m46.514245933s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993607952 (235h13m46.514431697s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bbyRMajztw9frbd1V1meid4vQU1AC343NtZrg6qe27HUyw6bQ): protocol: read tcp 188.166.225.170:57672->45.91.116.8:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993615226 (235h13m46.514438971s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993648048 (235h13m46.514471793s) [seg_pos={0 0}, upload=225259]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 10 14:01:35.993741707 (235h13m46.514565452s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 10 14:01:35.993749172 (235h13m46.514572917s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 10 14:01:36.289004295 (235h13m46.809828041s) [upload=225259]: Done uploading segments... May 10 14:01:36.289022049 (235h13m46.809845794s) [upload=225259]: Upload finished. err: <nil> Duration: 5.80621369s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [.................................X.X............X.....X.....] duration: 4.57287887s trace: https://jaeger.dev.storj.io/trace/35caab641616e4aa FAIL (2026-05-10 14:14:33 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.572897896s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.579139341s trace: https://jaeger.dev.storj.io/trace/6648ff2bad076ed9 FAIL (2026-05-10 14:13:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT, Piece ID: OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A) context deadline exceeded; piecestore: (Node ID: 1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, Piece ID: T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA) context deadline exceeded ======LOGS====== 21893s) [upload=225259]: Uploading segments... May 10 14:01:16.224858523 (235h13m26.745682258s) [upload=225259]: Got next segment. Inline: false May 10 14:01:16.224887649 (235h13m26.745711404s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 10 14:01:16.224892628 (235h13m26.745716373s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 10 14:01:16.458648925 (235h13m26.97947267s) [seg_pos={0 0}, upload=225259]: Begin upload segment... May 10 14:01:16.458694282 (235h13m26.979518027s) [seg_pos={0 0}, upload=225259]: Done begin upload segment. May 10 14:01:16.458731002 (235h13m26.979554777s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458759456 (235h13m26.979583202s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458843968 (235h13m26.979667713s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459127068 (235h13m26.979950813s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459464842 (235h13m26.980288587s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459745368 (235h13m26.980569113s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460016295 (235h13m26.98084004s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460259599 (235h13m26.981083344s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460587765 (235h13m26.98141151s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.462023354 (235h13m26.982847109s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.463113816 (235h13m26.983937561s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.464084608 (235h13m26.984908353s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.465045181 (235h13m26.985868926s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.466126985 (235h13m26.98695073s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467130310 (235h13m26.987954055s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467480458 (235h13m26.988304203s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:18.171956868 (235h13m28.692780613s) [upload=225259]: Next returned nil segment May 10 14:01:18.172017063 (235h13m28.692840808s) [upload=225259]: Waiting for error group managing segments... May 10 14:01:18.458161229 (235h13m28.978984994s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.463350482 (235h13m28.984174227s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.465465377 (235h13m28.986289122s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.467489280 (235h13m28.988313025s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.470505204 (235h13m28.991328949s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.476761352 (235h13m28.997585097s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.479218581 (235h13m29.000042326s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.487335090 (235h13m29.008158855s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.498784128 (235h13m29.019607873s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.504268754 (235h13m29.025092499s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:19.367580157 (235h13m29.888403903s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:24.805735031 (235h13m35.326558766s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992630767 (235h13m46.513454532s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992651637 (235h13m46.513475392s) [seg_pos={0 0}, upload=225259]: Satisfied cohort requirements after 13 pieces May 10 14:01:35.993179815 (235h13m46.51400356s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection May 10 14:01:35.993188822 (235h13m46.514012567s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993412679 (235h13m46.514236424s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: read tcp 188.166.225.170:44272->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993422178 (235h13m46.514245933s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993607952 (235h13m46.514431697s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bbyRMajztw9frbd1V1meid4vQU1AC343NtZrg6qe27HUyw6bQ): protocol: read tcp 188.166.225.170:57672->45.91.116.8:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993615226 (235h13m46.514438971s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993648048 (235h13m46.514471793s) [seg_pos={0 0}, upload=225259]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 10 14:01:35.993741707 (235h13m46.514565452s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 10 14:01:35.993749172 (235h13m46.514572917s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 10 14:01:36.289004295 (235h13m46.809828041s) [upload=225259]: Done uploading segments... May 10 14:01:36.289022049 (235h13m46.809845794s) [upload=225259]: Upload finished. err: <nil> Duration: 5.579161685s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 7.072628158s trace: https://jaeger.dev.storj.io/trace/57e77812227844e8 FAIL (2026-05-10 14:13:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.072697851s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [..................................X.X............X.....X....] duration: 4.016766283s trace: https://jaeger.dev.storj.io/trace/35caab641616e4aa FAIL (2026-05-10 14:13:44 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.016787563s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [...................................X.X............X.....X...] duration: 4.186628513s trace: https://jaeger.dev.storj.io/trace/35caab641616e4aa FAIL (2026-05-10 14:13:29 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.18665339s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 5.714103488s trace: https://jaeger.dev.storj.io/trace/6648ff2bad076ed9 FAIL (2026-05-10 14:13:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT, Piece ID: OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A) context deadline exceeded; piecestore: (Node ID: 1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, Piece ID: T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA) context deadline exceeded ======LOGS====== 21893s) [upload=225259]: Uploading segments... May 10 14:01:16.224858523 (235h13m26.745682258s) [upload=225259]: Got next segment. Inline: false May 10 14:01:16.224887649 (235h13m26.745711404s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 10 14:01:16.224892628 (235h13m26.745716373s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 10 14:01:16.458648925 (235h13m26.97947267s) [seg_pos={0 0}, upload=225259]: Begin upload segment... May 10 14:01:16.458694282 (235h13m26.979518027s) [seg_pos={0 0}, upload=225259]: Done begin upload segment. May 10 14:01:16.458731002 (235h13m26.979554777s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458759456 (235h13m26.979583202s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458843968 (235h13m26.979667713s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459127068 (235h13m26.979950813s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459464842 (235h13m26.980288587s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459745368 (235h13m26.980569113s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460016295 (235h13m26.98084004s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460259599 (235h13m26.981083344s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460587765 (235h13m26.98141151s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.462023354 (235h13m26.982847109s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.463113816 (235h13m26.983937561s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.464084608 (235h13m26.984908353s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.465045181 (235h13m26.985868926s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.466126985 (235h13m26.98695073s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467130310 (235h13m26.987954055s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467480458 (235h13m26.988304203s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:18.171956868 (235h13m28.692780613s) [upload=225259]: Next returned nil segment May 10 14:01:18.172017063 (235h13m28.692840808s) [upload=225259]: Waiting for error group managing segments... May 10 14:01:18.458161229 (235h13m28.978984994s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.463350482 (235h13m28.984174227s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.465465377 (235h13m28.986289122s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.467489280 (235h13m28.988313025s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.470505204 (235h13m28.991328949s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.476761352 (235h13m28.997585097s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.479218581 (235h13m29.000042326s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.487335090 (235h13m29.008158855s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.498784128 (235h13m29.019607873s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.504268754 (235h13m29.025092499s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:19.367580157 (235h13m29.888403903s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:24.805735031 (235h13m35.326558766s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992630767 (235h13m46.513454532s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992651637 (235h13m46.513475392s) [seg_pos={0 0}, upload=225259]: Satisfied cohort requirements after 13 pieces May 10 14:01:35.993179815 (235h13m46.51400356s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection May 10 14:01:35.993188822 (235h13m46.514012567s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993412679 (235h13m46.514236424s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: read tcp 188.166.225.170:44272->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993422178 (235h13m46.514245933s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993607952 (235h13m46.514431697s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bbyRMajztw9frbd1V1meid4vQU1AC343NtZrg6qe27HUyw6bQ): protocol: read tcp 188.166.225.170:57672->45.91.116.8:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993615226 (235h13m46.514438971s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993648048 (235h13m46.514471793s) [seg_pos={0 0}, upload=225259]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 10 14:01:35.993741707 (235h13m46.514565452s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 10 14:01:35.993749172 (235h13m46.514572917s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 10 14:01:36.289004295 (235h13m46.809828041s) [upload=225259]: Done uploading segments... May 10 14:01:36.289022049 (235h13m46.809845794s) [upload=225259]: Upload finished. err: <nil> Duration: 5.714130541s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 7.293683499s trace: https://jaeger.dev.storj.io/trace/57e77812227844e8 FAIL (2026-05-10 14:12:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.293711583s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [....................................X.X............X.....X..] duration: 3.957394288s trace: https://jaeger.dev.storj.io/trace/35caab641616e4aa FAIL (2026-05-10 14:12:23 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.957413154s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.747536995s trace: https://jaeger.dev.storj.io/trace/6648ff2bad076ed9 FAIL (2026-05-10 14:12:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT, Piece ID: OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A) context deadline exceeded; piecestore: (Node ID: 1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, Piece ID: T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA) context deadline exceeded ======LOGS====== 21893s) [upload=225259]: Uploading segments... May 10 14:01:16.224858523 (235h13m26.745682258s) [upload=225259]: Got next segment. Inline: false May 10 14:01:16.224887649 (235h13m26.745711404s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 10 14:01:16.224892628 (235h13m26.745716373s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 10 14:01:16.458648925 (235h13m26.97947267s) [seg_pos={0 0}, upload=225259]: Begin upload segment... May 10 14:01:16.458694282 (235h13m26.979518027s) [seg_pos={0 0}, upload=225259]: Done begin upload segment. May 10 14:01:16.458731002 (235h13m26.979554777s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458759456 (235h13m26.979583202s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458843968 (235h13m26.979667713s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459127068 (235h13m26.979950813s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459464842 (235h13m26.980288587s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459745368 (235h13m26.980569113s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460016295 (235h13m26.98084004s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460259599 (235h13m26.981083344s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460587765 (235h13m26.98141151s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.462023354 (235h13m26.982847109s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.463113816 (235h13m26.983937561s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.464084608 (235h13m26.984908353s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.465045181 (235h13m26.985868926s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.466126985 (235h13m26.98695073s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467130310 (235h13m26.987954055s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467480458 (235h13m26.988304203s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:18.171956868 (235h13m28.692780613s) [upload=225259]: Next returned nil segment May 10 14:01:18.172017063 (235h13m28.692840808s) [upload=225259]: Waiting for error group managing segments... May 10 14:01:18.458161229 (235h13m28.978984994s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.463350482 (235h13m28.984174227s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.465465377 (235h13m28.986289122s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.467489280 (235h13m28.988313025s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.470505204 (235h13m28.991328949s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.476761352 (235h13m28.997585097s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.479218581 (235h13m29.000042326s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.487335090 (235h13m29.008158855s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.498784128 (235h13m29.019607873s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.504268754 (235h13m29.025092499s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:19.367580157 (235h13m29.888403903s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:24.805735031 (235h13m35.326558766s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992630767 (235h13m46.513454532s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992651637 (235h13m46.513475392s) [seg_pos={0 0}, upload=225259]: Satisfied cohort requirements after 13 pieces May 10 14:01:35.993179815 (235h13m46.51400356s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection May 10 14:01:35.993188822 (235h13m46.514012567s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993412679 (235h13m46.514236424s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: read tcp 188.166.225.170:44272->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993422178 (235h13m46.514245933s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993607952 (235h13m46.514431697s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bbyRMajztw9frbd1V1meid4vQU1AC343NtZrg6qe27HUyw6bQ): protocol: read tcp 188.166.225.170:57672->45.91.116.8:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993615226 (235h13m46.514438971s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993648048 (235h13m46.514471793s) [seg_pos={0 0}, upload=225259]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 10 14:01:35.993741707 (235h13m46.514565452s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 10 14:01:35.993749172 (235h13m46.514572917s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 10 14:01:36.289004295 (235h13m46.809828041s) [upload=225259]: Done uploading segments... May 10 14:01:36.289022049 (235h13m46.809845794s) [upload=225259]: Upload finished. err: <nil> Duration: 5.747558747s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 7.467073852s trace: https://jaeger.dev.storj.io/trace/57e77812227844e8 FAIL (2026-05-10 14:11:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.467097197s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 7.008456151s trace: https://jaeger.dev.storj.io/trace/57e77812227844e8 FAIL (2026-05-10 14:11:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.008479655s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [.....................................X.X............X.....X.] duration: 3.927135794s trace: https://jaeger.dev.storj.io/trace/35caab641616e4aa FAIL (2026-05-10 14:11:20 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.92716035s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.77495368s trace: https://jaeger.dev.storj.io/trace/6648ff2bad076ed9 FAIL (2026-05-10 14:10:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT, Piece ID: OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A) context deadline exceeded; piecestore: (Node ID: 1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, Piece ID: T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA) context deadline exceeded ======LOGS====== 21893s) [upload=225259]: Uploading segments... May 10 14:01:16.224858523 (235h13m26.745682258s) [upload=225259]: Got next segment. Inline: false May 10 14:01:16.224887649 (235h13m26.745711404s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 10 14:01:16.224892628 (235h13m26.745716373s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 10 14:01:16.458648925 (235h13m26.97947267s) [seg_pos={0 0}, upload=225259]: Begin upload segment... May 10 14:01:16.458694282 (235h13m26.979518027s) [seg_pos={0 0}, upload=225259]: Done begin upload segment. May 10 14:01:16.458731002 (235h13m26.979554777s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458759456 (235h13m26.979583202s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458843968 (235h13m26.979667713s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459127068 (235h13m26.979950813s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459464842 (235h13m26.980288587s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459745368 (235h13m26.980569113s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460016295 (235h13m26.98084004s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460259599 (235h13m26.981083344s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460587765 (235h13m26.98141151s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.462023354 (235h13m26.982847109s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.463113816 (235h13m26.983937561s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.464084608 (235h13m26.984908353s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.465045181 (235h13m26.985868926s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.466126985 (235h13m26.98695073s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467130310 (235h13m26.987954055s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467480458 (235h13m26.988304203s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:18.171956868 (235h13m28.692780613s) [upload=225259]: Next returned nil segment May 10 14:01:18.172017063 (235h13m28.692840808s) [upload=225259]: Waiting for error group managing segments... May 10 14:01:18.458161229 (235h13m28.978984994s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.463350482 (235h13m28.984174227s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.465465377 (235h13m28.986289122s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.467489280 (235h13m28.988313025s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.470505204 (235h13m28.991328949s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.476761352 (235h13m28.997585097s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.479218581 (235h13m29.000042326s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.487335090 (235h13m29.008158855s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.498784128 (235h13m29.019607873s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.504268754 (235h13m29.025092499s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:19.367580157 (235h13m29.888403903s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:24.805735031 (235h13m35.326558766s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992630767 (235h13m46.513454532s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992651637 (235h13m46.513475392s) [seg_pos={0 0}, upload=225259]: Satisfied cohort requirements after 13 pieces May 10 14:01:35.993179815 (235h13m46.51400356s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection May 10 14:01:35.993188822 (235h13m46.514012567s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993412679 (235h13m46.514236424s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: read tcp 188.166.225.170:44272->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993422178 (235h13m46.514245933s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993607952 (235h13m46.514431697s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bbyRMajztw9frbd1V1meid4vQU1AC343NtZrg6qe27HUyw6bQ): protocol: read tcp 188.166.225.170:57672->45.91.116.8:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993615226 (235h13m46.514438971s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993648048 (235h13m46.514471793s) [seg_pos={0 0}, upload=225259]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 10 14:01:35.993741707 (235h13m46.514565452s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 10 14:01:35.993749172 (235h13m46.514572917s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 10 14:01:36.289004295 (235h13m46.809828041s) [upload=225259]: Done uploading segments... May 10 14:01:36.289022049 (235h13m46.809845794s) [upload=225259]: Upload finished. err: <nil> Duration: 5.774981803s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [......................................X.X............X.....X] duration: 5.001235174s trace: https://jaeger.dev.storj.io/trace/35caab641616e4aa FAIL (2026-05-10 14:10:25 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001257727s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 7.02472053s trace: https://jaeger.dev.storj.io/trace/57e77812227844e8 FAIL (2026-05-10 14:10:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.024738174s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.001639951s trace: https://jaeger.dev.storj.io/trace/57e77812227844e8 FAIL (2026-05-10 14:10:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001660209s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.884613659s trace: https://jaeger.dev.storj.io/trace/6648ff2bad076ed9 FAIL (2026-05-10 14:10:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT, Piece ID: OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A) context deadline exceeded; piecestore: (Node ID: 1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, Piece ID: T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA) context deadline exceeded ======LOGS====== 21893s) [upload=225259]: Uploading segments... May 10 14:01:16.224858523 (235h13m26.745682258s) [upload=225259]: Got next segment. Inline: false May 10 14:01:16.224887649 (235h13m26.745711404s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 10 14:01:16.224892628 (235h13m26.745716373s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 10 14:01:16.458648925 (235h13m26.97947267s) [seg_pos={0 0}, upload=225259]: Begin upload segment... May 10 14:01:16.458694282 (235h13m26.979518027s) [seg_pos={0 0}, upload=225259]: Done begin upload segment. May 10 14:01:16.458731002 (235h13m26.979554777s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458759456 (235h13m26.979583202s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458843968 (235h13m26.979667713s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459127068 (235h13m26.979950813s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459464842 (235h13m26.980288587s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459745368 (235h13m26.980569113s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460016295 (235h13m26.98084004s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460259599 (235h13m26.981083344s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460587765 (235h13m26.98141151s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.462023354 (235h13m26.982847109s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.463113816 (235h13m26.983937561s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.464084608 (235h13m26.984908353s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.465045181 (235h13m26.985868926s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.466126985 (235h13m26.98695073s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467130310 (235h13m26.987954055s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467480458 (235h13m26.988304203s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:18.171956868 (235h13m28.692780613s) [upload=225259]: Next returned nil segment May 10 14:01:18.172017063 (235h13m28.692840808s) [upload=225259]: Waiting for error group managing segments... May 10 14:01:18.458161229 (235h13m28.978984994s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.463350482 (235h13m28.984174227s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.465465377 (235h13m28.986289122s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.467489280 (235h13m28.988313025s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.470505204 (235h13m28.991328949s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.476761352 (235h13m28.997585097s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.479218581 (235h13m29.000042326s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.487335090 (235h13m29.008158855s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.498784128 (235h13m29.019607873s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.504268754 (235h13m29.025092499s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:19.367580157 (235h13m29.888403903s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:24.805735031 (235h13m35.326558766s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992630767 (235h13m46.513454532s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992651637 (235h13m46.513475392s) [seg_pos={0 0}, upload=225259]: Satisfied cohort requirements after 13 pieces May 10 14:01:35.993179815 (235h13m46.51400356s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection May 10 14:01:35.993188822 (235h13m46.514012567s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993412679 (235h13m46.514236424s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: read tcp 188.166.225.170:44272->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993422178 (235h13m46.514245933s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993607952 (235h13m46.514431697s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bbyRMajztw9frbd1V1meid4vQU1AC343NtZrg6qe27HUyw6bQ): protocol: read tcp 188.166.225.170:57672->45.91.116.8:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993615226 (235h13m46.514438971s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993648048 (235h13m46.514471793s) [seg_pos={0 0}, upload=225259]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 10 14:01:35.993741707 (235h13m46.514565452s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 10 14:01:35.993749172 (235h13m46.514572917s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 10 14:01:36.289004295 (235h13m46.809828041s) [upload=225259]: Done uploading segments... May 10 14:01:36.289022049 (235h13m46.809845794s) [upload=225259]: Upload finished. err: <nil> Duration: 5.884639509s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.......................................X.X............X.....] duration: 3.883786732s trace: https://jaeger.dev.storj.io/trace/2a5d3a36f283e319 FAIL (2026-05-10 14:09:19 UTC) orbiter warning: context deadline exceeded Duration: 3.883810828s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.760183042s trace: https://jaeger.dev.storj.io/trace/6648ff2bad076ed9 FAIL (2026-05-10 14:09:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT, Piece ID: OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A) context deadline exceeded; piecestore: (Node ID: 1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, Piece ID: T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA) context deadline exceeded ======LOGS====== 21893s) [upload=225259]: Uploading segments... May 10 14:01:16.224858523 (235h13m26.745682258s) [upload=225259]: Got next segment. Inline: false May 10 14:01:16.224887649 (235h13m26.745711404s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 10 14:01:16.224892628 (235h13m26.745716373s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 10 14:01:16.458648925 (235h13m26.97947267s) [seg_pos={0 0}, upload=225259]: Begin upload segment... May 10 14:01:16.458694282 (235h13m26.979518027s) [seg_pos={0 0}, upload=225259]: Done begin upload segment. May 10 14:01:16.458731002 (235h13m26.979554777s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458759456 (235h13m26.979583202s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458843968 (235h13m26.979667713s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459127068 (235h13m26.979950813s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459464842 (235h13m26.980288587s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459745368 (235h13m26.980569113s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460016295 (235h13m26.98084004s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460259599 (235h13m26.981083344s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460587765 (235h13m26.98141151s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.462023354 (235h13m26.982847109s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.463113816 (235h13m26.983937561s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.464084608 (235h13m26.984908353s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.465045181 (235h13m26.985868926s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.466126985 (235h13m26.98695073s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467130310 (235h13m26.987954055s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467480458 (235h13m26.988304203s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:18.171956868 (235h13m28.692780613s) [upload=225259]: Next returned nil segment May 10 14:01:18.172017063 (235h13m28.692840808s) [upload=225259]: Waiting for error group managing segments... May 10 14:01:18.458161229 (235h13m28.978984994s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.463350482 (235h13m28.984174227s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.465465377 (235h13m28.986289122s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.467489280 (235h13m28.988313025s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.470505204 (235h13m28.991328949s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.476761352 (235h13m28.997585097s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.479218581 (235h13m29.000042326s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.487335090 (235h13m29.008158855s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.498784128 (235h13m29.019607873s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.504268754 (235h13m29.025092499s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:19.367580157 (235h13m29.888403903s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:24.805735031 (235h13m35.326558766s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992630767 (235h13m46.513454532s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992651637 (235h13m46.513475392s) [seg_pos={0 0}, upload=225259]: Satisfied cohort requirements after 13 pieces May 10 14:01:35.993179815 (235h13m46.51400356s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection May 10 14:01:35.993188822 (235h13m46.514012567s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993412679 (235h13m46.514236424s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: read tcp 188.166.225.170:44272->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993422178 (235h13m46.514245933s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993607952 (235h13m46.514431697s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bbyRMajztw9frbd1V1meid4vQU1AC343NtZrg6qe27HUyw6bQ): protocol: read tcp 188.166.225.170:57672->45.91.116.8:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993615226 (235h13m46.514438971s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993648048 (235h13m46.514471793s) [seg_pos={0 0}, upload=225259]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 10 14:01:35.993741707 (235h13m46.514565452s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 10 14:01:35.993749172 (235h13m46.514572917s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 10 14:01:36.289004295 (235h13m46.809828041s) [upload=225259]: Done uploading segments... May 10 14:01:36.289022049 (235h13m46.809845794s) [upload=225259]: Upload finished. err: <nil> Duration: 5.760207489s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 5.687808977s trace: https://jaeger.dev.storj.io/trace/6648ff2bad076ed9 FAIL (2026-05-10 14:08:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT, Piece ID: OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A) context deadline exceeded; piecestore: (Node ID: 1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, Piece ID: T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA) context deadline exceeded ======LOGS====== 21893s) [upload=225259]: Uploading segments... May 10 14:01:16.224858523 (235h13m26.745682258s) [upload=225259]: Got next segment. Inline: false May 10 14:01:16.224887649 (235h13m26.745711404s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 10 14:01:16.224892628 (235h13m26.745716373s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 10 14:01:16.458648925 (235h13m26.97947267s) [seg_pos={0 0}, upload=225259]: Begin upload segment... May 10 14:01:16.458694282 (235h13m26.979518027s) [seg_pos={0 0}, upload=225259]: Done begin upload segment. May 10 14:01:16.458731002 (235h13m26.979554777s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458759456 (235h13m26.979583202s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458843968 (235h13m26.979667713s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459127068 (235h13m26.979950813s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459464842 (235h13m26.980288587s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459745368 (235h13m26.980569113s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460016295 (235h13m26.98084004s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460259599 (235h13m26.981083344s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460587765 (235h13m26.98141151s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.462023354 (235h13m26.982847109s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.463113816 (235h13m26.983937561s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.464084608 (235h13m26.984908353s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.465045181 (235h13m26.985868926s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.466126985 (235h13m26.98695073s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467130310 (235h13m26.987954055s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467480458 (235h13m26.988304203s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:18.171956868 (235h13m28.692780613s) [upload=225259]: Next returned nil segment May 10 14:01:18.172017063 (235h13m28.692840808s) [upload=225259]: Waiting for error group managing segments... May 10 14:01:18.458161229 (235h13m28.978984994s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.463350482 (235h13m28.984174227s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.465465377 (235h13m28.986289122s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.467489280 (235h13m28.988313025s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.470505204 (235h13m28.991328949s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.476761352 (235h13m28.997585097s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.479218581 (235h13m29.000042326s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.487335090 (235h13m29.008158855s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.498784128 (235h13m29.019607873s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.504268754 (235h13m29.025092499s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:19.367580157 (235h13m29.888403903s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:24.805735031 (235h13m35.326558766s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992630767 (235h13m46.513454532s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992651637 (235h13m46.513475392s) [seg_pos={0 0}, upload=225259]: Satisfied cohort requirements after 13 pieces May 10 14:01:35.993179815 (235h13m46.51400356s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection May 10 14:01:35.993188822 (235h13m46.514012567s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993412679 (235h13m46.514236424s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: read tcp 188.166.225.170:44272->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993422178 (235h13m46.514245933s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993607952 (235h13m46.514431697s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bbyRMajztw9frbd1V1meid4vQU1AC343NtZrg6qe27HUyw6bQ): protocol: read tcp 188.166.225.170:57672->45.91.116.8:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993615226 (235h13m46.514438971s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993648048 (235h13m46.514471793s) [seg_pos={0 0}, upload=225259]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 10 14:01:35.993741707 (235h13m46.514565452s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 10 14:01:35.993749172 (235h13m46.514572917s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 10 14:01:36.289004295 (235h13m46.809828041s) [upload=225259]: Done uploading segments... May 10 14:01:36.289022049 (235h13m46.809845794s) [upload=225259]: Upload finished. err: <nil> Duration: 5.687839296s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [........................................X.X............X....] duration: 5.000946791s trace: https://jaeger.dev.storj.io/trace/2a5d3a36f283e319 FAIL (2026-05-10 14:08:15 UTC) orbiter warning: context deadline exceeded Duration: 5.00097715s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.........................................X.X............X...] duration: 4.582287552s trace: https://jaeger.dev.storj.io/trace/2a5d3a36f283e319 FAIL (2026-05-10 14:07:43 UTC) orbiter warning: context deadline exceeded Duration: 4.58231265s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 5.606539192s trace: https://jaeger.dev.storj.io/trace/6648ff2bad076ed9 FAIL (2026-05-10 14:07:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT, Piece ID: OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A) context deadline exceeded; piecestore: (Node ID: 1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, Piece ID: T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA) context deadline exceeded ======LOGS====== 21893s) [upload=225259]: Uploading segments... May 10 14:01:16.224858523 (235h13m26.745682258s) [upload=225259]: Got next segment. Inline: false May 10 14:01:16.224887649 (235h13m26.745711404s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 10 14:01:16.224892628 (235h13m26.745716373s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 10 14:01:16.458648925 (235h13m26.97947267s) [seg_pos={0 0}, upload=225259]: Begin upload segment... May 10 14:01:16.458694282 (235h13m26.979518027s) [seg_pos={0 0}, upload=225259]: Done begin upload segment. May 10 14:01:16.458731002 (235h13m26.979554777s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458759456 (235h13m26.979583202s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458843968 (235h13m26.979667713s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459127068 (235h13m26.979950813s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459464842 (235h13m26.980288587s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459745368 (235h13m26.980569113s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460016295 (235h13m26.98084004s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460259599 (235h13m26.981083344s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460587765 (235h13m26.98141151s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.462023354 (235h13m26.982847109s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.463113816 (235h13m26.983937561s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.464084608 (235h13m26.984908353s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.465045181 (235h13m26.985868926s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.466126985 (235h13m26.98695073s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467130310 (235h13m26.987954055s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467480458 (235h13m26.988304203s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:18.171956868 (235h13m28.692780613s) [upload=225259]: Next returned nil segment May 10 14:01:18.172017063 (235h13m28.692840808s) [upload=225259]: Waiting for error group managing segments... May 10 14:01:18.458161229 (235h13m28.978984994s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.463350482 (235h13m28.984174227s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.465465377 (235h13m28.986289122s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.467489280 (235h13m28.988313025s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.470505204 (235h13m28.991328949s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.476761352 (235h13m28.997585097s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.479218581 (235h13m29.000042326s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.487335090 (235h13m29.008158855s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.498784128 (235h13m29.019607873s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.504268754 (235h13m29.025092499s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:19.367580157 (235h13m29.888403903s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:24.805735031 (235h13m35.326558766s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992630767 (235h13m46.513454532s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992651637 (235h13m46.513475392s) [seg_pos={0 0}, upload=225259]: Satisfied cohort requirements after 13 pieces May 10 14:01:35.993179815 (235h13m46.51400356s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection May 10 14:01:35.993188822 (235h13m46.514012567s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993412679 (235h13m46.514236424s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: read tcp 188.166.225.170:44272->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993422178 (235h13m46.514245933s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993607952 (235h13m46.514431697s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bbyRMajztw9frbd1V1meid4vQU1AC343NtZrg6qe27HUyw6bQ): protocol: read tcp 188.166.225.170:57672->45.91.116.8:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993615226 (235h13m46.514438971s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993648048 (235h13m46.514471793s) [seg_pos={0 0}, upload=225259]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 10 14:01:35.993741707 (235h13m46.514565452s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 10 14:01:35.993749172 (235h13m46.514572917s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 10 14:01:36.289004295 (235h13m46.809828041s) [upload=225259]: Done uploading segments... May 10 14:01:36.289022049 (235h13m46.809845794s) [upload=225259]: Upload finished. err: <nil> Duration: 5.606556917s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 7.31980763s trace: https://jaeger.dev.storj.io/trace/6648ff2bad076ed9 FAIL (2026-05-10 14:07:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT, Piece ID: OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A) context deadline exceeded; piecestore: (Node ID: 1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, Piece ID: T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA) context deadline exceeded ======LOGS====== 21893s) [upload=225259]: Uploading segments... May 10 14:01:16.224858523 (235h13m26.745682258s) [upload=225259]: Got next segment. Inline: false May 10 14:01:16.224887649 (235h13m26.745711404s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 10 14:01:16.224892628 (235h13m26.745716373s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 10 14:01:16.458648925 (235h13m26.97947267s) [seg_pos={0 0}, upload=225259]: Begin upload segment... May 10 14:01:16.458694282 (235h13m26.979518027s) [seg_pos={0 0}, upload=225259]: Done begin upload segment. May 10 14:01:16.458731002 (235h13m26.979554777s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458759456 (235h13m26.979583202s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458843968 (235h13m26.979667713s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459127068 (235h13m26.979950813s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459464842 (235h13m26.980288587s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459745368 (235h13m26.980569113s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460016295 (235h13m26.98084004s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460259599 (235h13m26.981083344s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460587765 (235h13m26.98141151s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.462023354 (235h13m26.982847109s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.463113816 (235h13m26.983937561s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.464084608 (235h13m26.984908353s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.465045181 (235h13m26.985868926s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.466126985 (235h13m26.98695073s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467130310 (235h13m26.987954055s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467480458 (235h13m26.988304203s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:18.171956868 (235h13m28.692780613s) [upload=225259]: Next returned nil segment May 10 14:01:18.172017063 (235h13m28.692840808s) [upload=225259]: Waiting for error group managing segments... May 10 14:01:18.458161229 (235h13m28.978984994s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.463350482 (235h13m28.984174227s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.465465377 (235h13m28.986289122s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.467489280 (235h13m28.988313025s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.470505204 (235h13m28.991328949s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.476761352 (235h13m28.997585097s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.479218581 (235h13m29.000042326s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.487335090 (235h13m29.008158855s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.498784128 (235h13m29.019607873s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.504268754 (235h13m29.025092499s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:19.367580157 (235h13m29.888403903s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:24.805735031 (235h13m35.326558766s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992630767 (235h13m46.513454532s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992651637 (235h13m46.513475392s) [seg_pos={0 0}, upload=225259]: Satisfied cohort requirements after 13 pieces May 10 14:01:35.993179815 (235h13m46.51400356s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection May 10 14:01:35.993188822 (235h13m46.514012567s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993412679 (235h13m46.514236424s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: read tcp 188.166.225.170:44272->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993422178 (235h13m46.514245933s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993607952 (235h13m46.514431697s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bbyRMajztw9frbd1V1meid4vQU1AC343NtZrg6qe27HUyw6bQ): protocol: read tcp 188.166.225.170:57672->45.91.116.8:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993615226 (235h13m46.514438971s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993648048 (235h13m46.514471793s) [seg_pos={0 0}, upload=225259]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 10 14:01:35.993741707 (235h13m46.514565452s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 10 14:01:35.993749172 (235h13m46.514572917s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 10 14:01:36.289004295 (235h13m46.809828041s) [upload=225259]: Done uploading segments... May 10 14:01:36.289022049 (235h13m46.809845794s) [upload=225259]: Upload finished. err: <nil> Duration: 7.319869819s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [..........................................X.X............X..] duration: 4.010826298s trace: https://jaeger.dev.storj.io/trace/2a5d3a36f283e319 FAIL (2026-05-10 14:06:38 UTC) orbiter warning: context deadline exceeded Duration: 4.010852659s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 5.793673922s trace: https://jaeger.dev.storj.io/trace/6648ff2bad076ed9 FAIL (2026-05-10 14:06:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT, Piece ID: OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A) context deadline exceeded; piecestore: (Node ID: 1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, Piece ID: T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA) context deadline exceeded ======LOGS====== 21893s) [upload=225259]: Uploading segments... May 10 14:01:16.224858523 (235h13m26.745682258s) [upload=225259]: Got next segment. Inline: false May 10 14:01:16.224887649 (235h13m26.745711404s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 10 14:01:16.224892628 (235h13m26.745716373s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 10 14:01:16.458648925 (235h13m26.97947267s) [seg_pos={0 0}, upload=225259]: Begin upload segment... May 10 14:01:16.458694282 (235h13m26.979518027s) [seg_pos={0 0}, upload=225259]: Done begin upload segment. May 10 14:01:16.458731002 (235h13m26.979554777s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458759456 (235h13m26.979583202s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458843968 (235h13m26.979667713s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459127068 (235h13m26.979950813s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459464842 (235h13m26.980288587s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459745368 (235h13m26.980569113s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460016295 (235h13m26.98084004s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460259599 (235h13m26.981083344s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460587765 (235h13m26.98141151s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.462023354 (235h13m26.982847109s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.463113816 (235h13m26.983937561s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.464084608 (235h13m26.984908353s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.465045181 (235h13m26.985868926s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.466126985 (235h13m26.98695073s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467130310 (235h13m26.987954055s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467480458 (235h13m26.988304203s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:18.171956868 (235h13m28.692780613s) [upload=225259]: Next returned nil segment May 10 14:01:18.172017063 (235h13m28.692840808s) [upload=225259]: Waiting for error group managing segments... May 10 14:01:18.458161229 (235h13m28.978984994s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.463350482 (235h13m28.984174227s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.465465377 (235h13m28.986289122s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.467489280 (235h13m28.988313025s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.470505204 (235h13m28.991328949s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.476761352 (235h13m28.997585097s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.479218581 (235h13m29.000042326s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.487335090 (235h13m29.008158855s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.498784128 (235h13m29.019607873s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.504268754 (235h13m29.025092499s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:19.367580157 (235h13m29.888403903s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:24.805735031 (235h13m35.326558766s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992630767 (235h13m46.513454532s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992651637 (235h13m46.513475392s) [seg_pos={0 0}, upload=225259]: Satisfied cohort requirements after 13 pieces May 10 14:01:35.993179815 (235h13m46.51400356s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection May 10 14:01:35.993188822 (235h13m46.514012567s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993412679 (235h13m46.514236424s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: read tcp 188.166.225.170:44272->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993422178 (235h13m46.514245933s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993607952 (235h13m46.514431697s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bbyRMajztw9frbd1V1meid4vQU1AC343NtZrg6qe27HUyw6bQ): protocol: read tcp 188.166.225.170:57672->45.91.116.8:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993615226 (235h13m46.514438971s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993648048 (235h13m46.514471793s) [seg_pos={0 0}, upload=225259]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 10 14:01:35.993741707 (235h13m46.514565452s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 10 14:01:35.993749172 (235h13m46.514572917s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 10 14:01:36.289004295 (235h13m46.809828041s) [upload=225259]: Done uploading segments... May 10 14:01:36.289022049 (235h13m46.809845794s) [upload=225259]: Upload finished. err: <nil> Duration: 5.793697608s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 7.603524174s trace: https://jaeger.dev.storj.io/trace/6648ff2bad076ed9 FAIL (2026-05-10 14:05:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT, Piece ID: OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A) context deadline exceeded; piecestore: (Node ID: 1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, Piece ID: T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA) context deadline exceeded ======LOGS====== 21893s) [upload=225259]: Uploading segments... May 10 14:01:16.224858523 (235h13m26.745682258s) [upload=225259]: Got next segment. Inline: false May 10 14:01:16.224887649 (235h13m26.745711404s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 10 14:01:16.224892628 (235h13m26.745716373s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 10 14:01:16.458648925 (235h13m26.97947267s) [seg_pos={0 0}, upload=225259]: Begin upload segment... May 10 14:01:16.458694282 (235h13m26.979518027s) [seg_pos={0 0}, upload=225259]: Done begin upload segment. May 10 14:01:16.458731002 (235h13m26.979554777s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458759456 (235h13m26.979583202s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458843968 (235h13m26.979667713s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459127068 (235h13m26.979950813s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459464842 (235h13m26.980288587s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459745368 (235h13m26.980569113s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460016295 (235h13m26.98084004s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460259599 (235h13m26.981083344s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460587765 (235h13m26.98141151s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.462023354 (235h13m26.982847109s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.463113816 (235h13m26.983937561s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.464084608 (235h13m26.984908353s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.465045181 (235h13m26.985868926s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.466126985 (235h13m26.98695073s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467130310 (235h13m26.987954055s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467480458 (235h13m26.988304203s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:18.171956868 (235h13m28.692780613s) [upload=225259]: Next returned nil segment May 10 14:01:18.172017063 (235h13m28.692840808s) [upload=225259]: Waiting for error group managing segments... May 10 14:01:18.458161229 (235h13m28.978984994s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.463350482 (235h13m28.984174227s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.465465377 (235h13m28.986289122s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.467489280 (235h13m28.988313025s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.470505204 (235h13m28.991328949s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.476761352 (235h13m28.997585097s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.479218581 (235h13m29.000042326s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.487335090 (235h13m29.008158855s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.498784128 (235h13m29.019607873s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.504268754 (235h13m29.025092499s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:19.367580157 (235h13m29.888403903s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:24.805735031 (235h13m35.326558766s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992630767 (235h13m46.513454532s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992651637 (235h13m46.513475392s) [seg_pos={0 0}, upload=225259]: Satisfied cohort requirements after 13 pieces May 10 14:01:35.993179815 (235h13m46.51400356s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection May 10 14:01:35.993188822 (235h13m46.514012567s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993412679 (235h13m46.514236424s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: read tcp 188.166.225.170:44272->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993422178 (235h13m46.514245933s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993607952 (235h13m46.514431697s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bbyRMajztw9frbd1V1meid4vQU1AC343NtZrg6qe27HUyw6bQ): protocol: read tcp 188.166.225.170:57672->45.91.116.8:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993615226 (235h13m46.514438971s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993648048 (235h13m46.514471793s) [seg_pos={0 0}, upload=225259]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 10 14:01:35.993741707 (235h13m46.514565452s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 10 14:01:35.993749172 (235h13m46.514572917s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 10 14:01:36.289004295 (235h13m46.809828041s) [upload=225259]: Done uploading segments... May 10 14:01:36.289022049 (235h13m46.809845794s) [upload=225259]: Upload finished. err: <nil> Duration: 7.603548439s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [...........................................X.X............X.] duration: 3.941090123s trace: https://jaeger.dev.storj.io/trace/2a5d3a36f283e319 FAIL (2026-05-10 14:05:26 UTC) orbiter warning: context deadline exceeded Duration: 3.941118367s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.734007766s trace: https://jaeger.dev.storj.io/trace/6648ff2bad076ed9 FAIL (2026-05-10 14:05:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT, Piece ID: OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A) context deadline exceeded; piecestore: (Node ID: 1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, Piece ID: T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA) context deadline exceeded ======LOGS====== 21893s) [upload=225259]: Uploading segments... May 10 14:01:16.224858523 (235h13m26.745682258s) [upload=225259]: Got next segment. Inline: false May 10 14:01:16.224887649 (235h13m26.745711404s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 10 14:01:16.224892628 (235h13m26.745716373s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 10 14:01:16.458648925 (235h13m26.97947267s) [seg_pos={0 0}, upload=225259]: Begin upload segment... May 10 14:01:16.458694282 (235h13m26.979518027s) [seg_pos={0 0}, upload=225259]: Done begin upload segment. May 10 14:01:16.458731002 (235h13m26.979554777s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458759456 (235h13m26.979583202s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458843968 (235h13m26.979667713s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459127068 (235h13m26.979950813s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459464842 (235h13m26.980288587s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459745368 (235h13m26.980569113s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460016295 (235h13m26.98084004s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460259599 (235h13m26.981083344s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460587765 (235h13m26.98141151s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.462023354 (235h13m26.982847109s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.463113816 (235h13m26.983937561s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.464084608 (235h13m26.984908353s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.465045181 (235h13m26.985868926s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.466126985 (235h13m26.98695073s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467130310 (235h13m26.987954055s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467480458 (235h13m26.988304203s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:18.171956868 (235h13m28.692780613s) [upload=225259]: Next returned nil segment May 10 14:01:18.172017063 (235h13m28.692840808s) [upload=225259]: Waiting for error group managing segments... May 10 14:01:18.458161229 (235h13m28.978984994s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.463350482 (235h13m28.984174227s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.465465377 (235h13m28.986289122s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.467489280 (235h13m28.988313025s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.470505204 (235h13m28.991328949s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.476761352 (235h13m28.997585097s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.479218581 (235h13m29.000042326s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.487335090 (235h13m29.008158855s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.498784128 (235h13m29.019607873s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.504268754 (235h13m29.025092499s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:19.367580157 (235h13m29.888403903s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:24.805735031 (235h13m35.326558766s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992630767 (235h13m46.513454532s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992651637 (235h13m46.513475392s) [seg_pos={0 0}, upload=225259]: Satisfied cohort requirements after 13 pieces May 10 14:01:35.993179815 (235h13m46.51400356s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection May 10 14:01:35.993188822 (235h13m46.514012567s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993412679 (235h13m46.514236424s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: read tcp 188.166.225.170:44272->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993422178 (235h13m46.514245933s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993607952 (235h13m46.514431697s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bbyRMajztw9frbd1V1meid4vQU1AC343NtZrg6qe27HUyw6bQ): protocol: read tcp 188.166.225.170:57672->45.91.116.8:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993615226 (235h13m46.514438971s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993648048 (235h13m46.514471793s) [seg_pos={0 0}, upload=225259]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 10 14:01:35.993741707 (235h13m46.514565452s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 10 14:01:35.993749172 (235h13m46.514572917s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 10 14:01:36.289004295 (235h13m46.809828041s) [upload=225259]: Done uploading segments... May 10 14:01:36.289022049 (235h13m46.809845794s) [upload=225259]: Upload finished. err: <nil> Duration: 5.734034266s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [............................................X.X............X] duration: 5.000879549s trace: https://jaeger.dev.storj.io/trace/2a5d3a36f283e319 FAIL (2026-05-10 14:04:45 UTC) orbiter warning: context deadline exceeded Duration: 5.000897122s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.955125194s trace: https://jaeger.dev.storj.io/trace/6648ff2bad076ed9 FAIL (2026-05-10 14:04:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT, Piece ID: OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A) context deadline exceeded; piecestore: (Node ID: 1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, Piece ID: T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA) context deadline exceeded ======LOGS====== 21893s) [upload=225259]: Uploading segments... May 10 14:01:16.224858523 (235h13m26.745682258s) [upload=225259]: Got next segment. Inline: false May 10 14:01:16.224887649 (235h13m26.745711404s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 10 14:01:16.224892628 (235h13m26.745716373s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 10 14:01:16.458648925 (235h13m26.97947267s) [seg_pos={0 0}, upload=225259]: Begin upload segment... May 10 14:01:16.458694282 (235h13m26.979518027s) [seg_pos={0 0}, upload=225259]: Done begin upload segment. May 10 14:01:16.458731002 (235h13m26.979554777s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458759456 (235h13m26.979583202s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458843968 (235h13m26.979667713s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459127068 (235h13m26.979950813s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459464842 (235h13m26.980288587s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459745368 (235h13m26.980569113s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460016295 (235h13m26.98084004s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460259599 (235h13m26.981083344s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460587765 (235h13m26.98141151s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.462023354 (235h13m26.982847109s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.463113816 (235h13m26.983937561s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.464084608 (235h13m26.984908353s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.465045181 (235h13m26.985868926s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.466126985 (235h13m26.98695073s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467130310 (235h13m26.987954055s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467480458 (235h13m26.988304203s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:18.171956868 (235h13m28.692780613s) [upload=225259]: Next returned nil segment May 10 14:01:18.172017063 (235h13m28.692840808s) [upload=225259]: Waiting for error group managing segments... May 10 14:01:18.458161229 (235h13m28.978984994s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.463350482 (235h13m28.984174227s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.465465377 (235h13m28.986289122s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.467489280 (235h13m28.988313025s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.470505204 (235h13m28.991328949s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.476761352 (235h13m28.997585097s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.479218581 (235h13m29.000042326s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.487335090 (235h13m29.008158855s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.498784128 (235h13m29.019607873s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.504268754 (235h13m29.025092499s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:19.367580157 (235h13m29.888403903s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:24.805735031 (235h13m35.326558766s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992630767 (235h13m46.513454532s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992651637 (235h13m46.513475392s) [seg_pos={0 0}, upload=225259]: Satisfied cohort requirements after 13 pieces May 10 14:01:35.993179815 (235h13m46.51400356s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection May 10 14:01:35.993188822 (235h13m46.514012567s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993412679 (235h13m46.514236424s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: read tcp 188.166.225.170:44272->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993422178 (235h13m46.514245933s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993607952 (235h13m46.514431697s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bbyRMajztw9frbd1V1meid4vQU1AC343NtZrg6qe27HUyw6bQ): protocol: read tcp 188.166.225.170:57672->45.91.116.8:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993615226 (235h13m46.514438971s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993648048 (235h13m46.514471793s) [seg_pos={0 0}, upload=225259]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 10 14:01:35.993741707 (235h13m46.514565452s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 10 14:01:35.993749172 (235h13m46.514572917s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 10 14:01:36.289004295 (235h13m46.809828041s) [upload=225259]: Done uploading segments... May 10 14:01:36.289022049 (235h13m46.809845794s) [upload=225259]: Upload finished. err: <nil> Duration: 5.955150201s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.761875884s trace: https://jaeger.dev.storj.io/trace/6648ff2bad076ed9 FAIL (2026-05-10 14:04:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT, Piece ID: OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A) context deadline exceeded; piecestore: (Node ID: 1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, Piece ID: T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA) context deadline exceeded ======LOGS====== 21893s) [upload=225259]: Uploading segments... May 10 14:01:16.224858523 (235h13m26.745682258s) [upload=225259]: Got next segment. Inline: false May 10 14:01:16.224887649 (235h13m26.745711404s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 10 14:01:16.224892628 (235h13m26.745716373s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 10 14:01:16.458648925 (235h13m26.97947267s) [seg_pos={0 0}, upload=225259]: Begin upload segment... May 10 14:01:16.458694282 (235h13m26.979518027s) [seg_pos={0 0}, upload=225259]: Done begin upload segment. May 10 14:01:16.458731002 (235h13m26.979554777s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458759456 (235h13m26.979583202s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458843968 (235h13m26.979667713s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459127068 (235h13m26.979950813s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459464842 (235h13m26.980288587s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459745368 (235h13m26.980569113s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460016295 (235h13m26.98084004s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460259599 (235h13m26.981083344s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460587765 (235h13m26.98141151s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.462023354 (235h13m26.982847109s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.463113816 (235h13m26.983937561s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.464084608 (235h13m26.984908353s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.465045181 (235h13m26.985868926s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.466126985 (235h13m26.98695073s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467130310 (235h13m26.987954055s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467480458 (235h13m26.988304203s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:18.171956868 (235h13m28.692780613s) [upload=225259]: Next returned nil segment May 10 14:01:18.172017063 (235h13m28.692840808s) [upload=225259]: Waiting for error group managing segments... May 10 14:01:18.458161229 (235h13m28.978984994s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.463350482 (235h13m28.984174227s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.465465377 (235h13m28.986289122s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.467489280 (235h13m28.988313025s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.470505204 (235h13m28.991328949s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.476761352 (235h13m28.997585097s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.479218581 (235h13m29.000042326s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.487335090 (235h13m29.008158855s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.498784128 (235h13m29.019607873s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.504268754 (235h13m29.025092499s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:19.367580157 (235h13m29.888403903s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:24.805735031 (235h13m35.326558766s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992630767 (235h13m46.513454532s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992651637 (235h13m46.513475392s) [seg_pos={0 0}, upload=225259]: Satisfied cohort requirements after 13 pieces May 10 14:01:35.993179815 (235h13m46.51400356s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection May 10 14:01:35.993188822 (235h13m46.514012567s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993412679 (235h13m46.514236424s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: read tcp 188.166.225.170:44272->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993422178 (235h13m46.514245933s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993607952 (235h13m46.514431697s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bbyRMajztw9frbd1V1meid4vQU1AC343NtZrg6qe27HUyw6bQ): protocol: read tcp 188.166.225.170:57672->45.91.116.8:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993615226 (235h13m46.514438971s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993648048 (235h13m46.514471793s) [seg_pos={0 0}, upload=225259]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 10 14:01:35.993741707 (235h13m46.514565452s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 10 14:01:35.993749172 (235h13m46.514572917s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 10 14:01:36.289004295 (235h13m46.809828041s) [upload=225259]: Done uploading segments... May 10 14:01:36.289022049 (235h13m46.809845794s) [upload=225259]: Upload finished. err: <nil> Duration: 5.761904439s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.............................................X.X............] duration: 4.054624726s trace: https://jaeger.dev.storj.io/trace/2a36fe89f5981837 FAIL (2026-05-10 14:03:37 UTC) orbiter warning: context deadline exceeded Duration: 4.054649985s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.349731055s trace: https://jaeger.dev.storj.io/trace/6648ff2bad076ed9 FAIL (2026-05-10 14:03:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT, Piece ID: OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A) context deadline exceeded; piecestore: (Node ID: 1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, Piece ID: T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA) context deadline exceeded ======LOGS====== 21893s) [upload=225259]: Uploading segments... May 10 14:01:16.224858523 (235h13m26.745682258s) [upload=225259]: Got next segment. Inline: false May 10 14:01:16.224887649 (235h13m26.745711404s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 10 14:01:16.224892628 (235h13m26.745716373s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 10 14:01:16.458648925 (235h13m26.97947267s) [seg_pos={0 0}, upload=225259]: Begin upload segment... May 10 14:01:16.458694282 (235h13m26.979518027s) [seg_pos={0 0}, upload=225259]: Done begin upload segment. May 10 14:01:16.458731002 (235h13m26.979554777s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458759456 (235h13m26.979583202s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458843968 (235h13m26.979667713s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459127068 (235h13m26.979950813s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459464842 (235h13m26.980288587s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459745368 (235h13m26.980569113s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460016295 (235h13m26.98084004s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460259599 (235h13m26.981083344s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460587765 (235h13m26.98141151s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.462023354 (235h13m26.982847109s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.463113816 (235h13m26.983937561s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.464084608 (235h13m26.984908353s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.465045181 (235h13m26.985868926s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.466126985 (235h13m26.98695073s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467130310 (235h13m26.987954055s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467480458 (235h13m26.988304203s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:18.171956868 (235h13m28.692780613s) [upload=225259]: Next returned nil segment May 10 14:01:18.172017063 (235h13m28.692840808s) [upload=225259]: Waiting for error group managing segments... May 10 14:01:18.458161229 (235h13m28.978984994s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.463350482 (235h13m28.984174227s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.465465377 (235h13m28.986289122s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.467489280 (235h13m28.988313025s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.470505204 (235h13m28.991328949s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.476761352 (235h13m28.997585097s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.479218581 (235h13m29.000042326s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.487335090 (235h13m29.008158855s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.498784128 (235h13m29.019607873s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.504268754 (235h13m29.025092499s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:19.367580157 (235h13m29.888403903s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:24.805735031 (235h13m35.326558766s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992630767 (235h13m46.513454532s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992651637 (235h13m46.513475392s) [seg_pos={0 0}, upload=225259]: Satisfied cohort requirements after 13 pieces May 10 14:01:35.993179815 (235h13m46.51400356s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection May 10 14:01:35.993188822 (235h13m46.514012567s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993412679 (235h13m46.514236424s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: read tcp 188.166.225.170:44272->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993422178 (235h13m46.514245933s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993607952 (235h13m46.514431697s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bbyRMajztw9frbd1V1meid4vQU1AC343NtZrg6qe27HUyw6bQ): protocol: read tcp 188.166.225.170:57672->45.91.116.8:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993615226 (235h13m46.514438971s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993648048 (235h13m46.514471793s) [seg_pos={0 0}, upload=225259]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 10 14:01:35.993741707 (235h13m46.514565452s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 10 14:01:35.993749172 (235h13m46.514572917s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 10 14:01:36.289004295 (235h13m46.809828041s) [upload=225259]: Done uploading segments... May 10 14:01:36.289022049 (235h13m46.809845794s) [upload=225259]: Upload finished. err: <nil> Duration: 6.349758306s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..............................................X.X...........] duration: 4.549638321s trace: https://jaeger.dev.storj.io/trace/2a36fe89f5981837 FAIL (2026-05-10 14:03:10 UTC) orbiter warning: context deadline exceeded Duration: 4.54971133s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...............................................X.X..........] duration: 4.03283708s trace: https://jaeger.dev.storj.io/trace/2a36fe89f5981837 FAIL (2026-05-10 14:02:13 UTC) orbiter warning: context deadline exceeded Duration: 4.032857429s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 14.575324891s trace: https://jaeger.dev.storj.io/trace/6648ff2bad076ed9 FAIL (2026-05-10 14:02:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT, Piece ID: OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A) context deadline exceeded; piecestore: (Node ID: 1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, Piece ID: T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA) context deadline exceeded ======LOGS====== 21893s) [upload=225259]: Uploading segments... May 10 14:01:16.224858523 (235h13m26.745682258s) [upload=225259]: Got next segment. Inline: false May 10 14:01:16.224887649 (235h13m26.745711404s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 10 14:01:16.224892628 (235h13m26.745716373s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 10 14:01:16.458648925 (235h13m26.97947267s) [seg_pos={0 0}, upload=225259]: Begin upload segment... May 10 14:01:16.458694282 (235h13m26.979518027s) [seg_pos={0 0}, upload=225259]: Done begin upload segment. May 10 14:01:16.458731002 (235h13m26.979554777s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458759456 (235h13m26.979583202s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458843968 (235h13m26.979667713s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459127068 (235h13m26.979950813s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459464842 (235h13m26.980288587s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459745368 (235h13m26.980569113s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460016295 (235h13m26.98084004s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460259599 (235h13m26.981083344s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460587765 (235h13m26.98141151s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.462023354 (235h13m26.982847109s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.463113816 (235h13m26.983937561s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.464084608 (235h13m26.984908353s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.465045181 (235h13m26.985868926s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.466126985 (235h13m26.98695073s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467130310 (235h13m26.987954055s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467480458 (235h13m26.988304203s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:18.171956868 (235h13m28.692780613s) [upload=225259]: Next returned nil segment May 10 14:01:18.172017063 (235h13m28.692840808s) [upload=225259]: Waiting for error group managing segments... May 10 14:01:18.458161229 (235h13m28.978984994s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.463350482 (235h13m28.984174227s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.465465377 (235h13m28.986289122s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.467489280 (235h13m28.988313025s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.470505204 (235h13m28.991328949s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.476761352 (235h13m28.997585097s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.479218581 (235h13m29.000042326s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.487335090 (235h13m29.008158855s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.498784128 (235h13m29.019607873s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.504268754 (235h13m29.025092499s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:19.367580157 (235h13m29.888403903s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:24.805735031 (235h13m35.326558766s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992630767 (235h13m46.513454532s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992651637 (235h13m46.513475392s) [seg_pos={0 0}, upload=225259]: Satisfied cohort requirements after 13 pieces May 10 14:01:35.993179815 (235h13m46.51400356s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection May 10 14:01:35.993188822 (235h13m46.514012567s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993412679 (235h13m46.514236424s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: read tcp 188.166.225.170:44272->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993422178 (235h13m46.514245933s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993607952 (235h13m46.514431697s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bbyRMajztw9frbd1V1meid4vQU1AC343NtZrg6qe27HUyw6bQ): protocol: read tcp 188.166.225.170:57672->45.91.116.8:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993615226 (235h13m46.514438971s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993648048 (235h13m46.514471793s) [seg_pos={0 0}, upload=225259]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 10 14:01:35.993741707 (235h13m46.514565452s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 10 14:01:35.993749172 (235h13m46.514572917s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 10 14:01:36.289004295 (235h13m46.809828041s) [upload=225259]: Done uploading segments... May 10 14:01:36.289022049 (235h13m46.809845794s) [upload=225259]: Upload finished. err: <nil> Duration: 14.575354127s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [................................................X.X.........] duration: 3.947945445s trace: https://jaeger.dev.storj.io/trace/2a36fe89f5981837 FAIL (2026-05-10 14:01:49 UTC) orbiter warning: context deadline exceeded Duration: 3.947962627s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.002819384s trace: https://jaeger.dev.storj.io/trace/6648ff2bad076ed9 FAIL (2026-05-10 14:01:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jzasZMDY1SsvNgPbDJ2LqhBiHbLnGgHcnXEkUZ4a6UDoCpgsT, Piece ID: OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A) context deadline exceeded; piecestore: (Node ID: 1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, Piece ID: T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA) context deadline exceeded ======LOGS====== 21893s) [upload=225259]: Uploading segments... May 10 14:01:16.224858523 (235h13m26.745682258s) [upload=225259]: Got next segment. Inline: false May 10 14:01:16.224887649 (235h13m26.745711404s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 10 14:01:16.224892628 (235h13m26.745716373s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 10 14:01:16.458648925 (235h13m26.97947267s) [seg_pos={0 0}, upload=225259]: Begin upload segment... May 10 14:01:16.458694282 (235h13m26.979518027s) [seg_pos={0 0}, upload=225259]: Done begin upload segment. May 10 14:01:16.458731002 (235h13m26.979554777s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458759456 (235h13m26.979583202s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.458843968 (235h13m26.979667713s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459127068 (235h13m26.979950813s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459464842 (235h13m26.980288587s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.459745368 (235h13m26.980569113s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460016295 (235h13m26.98084004s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460259599 (235h13m26.981083344s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.460587765 (235h13m26.98141151s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.462023354 (235h13m26.982847109s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.463113816 (235h13m26.983937561s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.464084608 (235h13m26.984908353s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.465045181 (235h13m26.985868926s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.466126985 (235h13m26.98695073s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467130310 (235h13m26.987954055s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:16.467480458 (235h13m26.988304203s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Uploading piece... May 10 14:01:18.171956868 (235h13m28.692780613s) [upload=225259]: Next returned nil segment May 10 14:01:18.172017063 (235h13m28.692840808s) [upload=225259]: Waiting for error group managing segments... May 10 14:01:18.458161229 (235h13m28.978984994s) [noise=true , address=145.79.227.39:4016 , piece_id=WYNFXLW5YHTQ77GFNVG6DSSNHM65TCLDZLHMHN24M4TJ72J77L7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.463350482 (235h13m28.984174227s) [noise=true , address=145.79.227.5:4227 , piece_id=IVPLAT5U3FCK5ZE4H635SOILFPPL3BVARLL2X4V6DLFDFO6RFIXA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.465465377 (235h13m28.986289122s) [noise=true , address=145.79.227.19:4077 , piece_id=D7FZF5U4CDSTLAEZZJSOGMYKKMHCEZETWPPUPYB6IS4MCZAJKI4Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.467489280 (235h13m28.988313025s) [noise=true , address=145.79.227.7:4295 , piece_id=HMS5EPXF56BWO3LSSC2E7APE6Q5EOQIZPOMN3I3BWIAHT2GTRAGQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.470505204 (235h13m28.991328949s) [noise=true , address=145.79.227.36:4521 , piece_id=I4364ZTQCMJ3OKC244CYNCE7FTWUFMAVFC5D5HPD7WXV6X47A5FA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.476761352 (235h13m28.997585097s) [noise=true , address=145.79.227.35:4483 , piece_id=3XXB2W24ZTPROFOQSXVXDWVZGBOBO27BXXNPHR2435V22DUT4OGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.479218581 (235h13m29.000042326s) [noise=true , address=145.79.227.47:4269 , piece_id=H2VL3PJ4DRT6WRDH5IENVPPHCIUSW7BSDWMC5Q2XXNTGFHB6E5BA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.487335090 (235h13m29.008158855s) [noise=true , address=145.79.227.51:4409 , piece_id=AQH33YG66KJSTKN7UKB7VL63TGTLYH7PBHUBGNL2VEBURWRUJMZQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.498784128 (235h13m29.019607873s) [noise=true , address=145.79.227.24:4587 , piece_id=SPOSDKPJDIEPTVJEWKEPMFXPSYH4GDB5WGRVEGTB2QEKX7JGXJ5Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:18.504268754 (235h13m29.025092499s) [noise=true , address=77.90.31.19:4845 , piece_id=B765JJPUU3FV5AMK66WZ3IJ7RY4QZJBG6XT3CMUOK3QPEAKUPJUQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:19.367580157 (235h13m29.888403903s) [noise=true , address=145.79.227.9:4369 , piece_id=I4SQRRMM5T4BALW5SLKVSNKFRJ2RVBNWRA655DJC4II72SHGTAGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:24.805735031 (235h13m35.326558766s) [noise=true , address=45.91.116.4:4953 , piece_id=OPUMX2OD5BXLVEMJJ4BDX6WK75TC42LNWJO5NBB45OP446ETJL7A, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992630767 (235h13m46.513454532s) [noise=true , address=45.91.116.7:4012 , piece_id=T6U2R7PBHYMNU5SSQGSJWDOFRKQESQIJDIKRSHP6IP2VMFX7TYGA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: <nil> May 10 14:01:35.992651637 (235h13m46.513475392s) [seg_pos={0 0}, upload=225259]: Satisfied cohort requirements after 13 pieces May 10 14:01:35.993179815 (235h13m46.51400356s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12esv4CJj8pQep6GYNbWEEnuhgyPoNUNanWqQDW3kEUwqks9eNs): protocol: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54904->45.91.116.6:4989: use of closed network connection May 10 14:01:35.993188822 (235h13m46.514012567s) [noise=true , address=45.91.116.6:4989 , piece_id=MTOT7ZFNET4CGHUQ4HZTNMR5GLFVYL2ALW4AFRHHJCMLBJKLX66Q, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993412679 (235h13m46.514236424s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: read tcp 188.166.225.170:44272->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993422178 (235h13m46.514245933s) [noise=true , address=45.91.116.5:4299 , piece_id=2PF4DOX24VUNBWJPCS6SONGUDLL27G6EH5ASHZT67NSB3BA4VCPQ, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993607952 (235h13m46.514431697s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bbyRMajztw9frbd1V1meid4vQU1AC343NtZrg6qe27HUyw6bQ): protocol: read tcp 188.166.225.170:57672->45.91.116.8:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 10 14:01:35.993615226 (235h13m46.514438971s) [noise=true , address=45.91.116.8:4049 , piece_id=UV6YT64LYO3U724IMPOGGWL2XCTQ4DLUUBAJ5MDI5KR4BLM2JQRA, seg_pos={0 0}, upload=225259]: Long tail cancellation due to optimal threshold May 10 14:01:35.993648048 (235h13m46.514471793s) [seg_pos={0 0}, upload=225259]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 10 14:01:35.993741707 (235h13m46.514565452s) [upload=225259]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 10 14:01:35.993749172 (235h13m46.514572917s) [upload=225259]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 10 14:01:36.289004295 (235h13m46.809828041s) [upload=225259]: Done uploading segments... May 10 14:01:36.289022049 (235h13m46.809845794s) [upload=225259]: Upload finished. err: <nil> Duration: 30.002840444s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.................................................X.X........] duration: 4.18998335s trace: https://jaeger.dev.storj.io/trace/2a36fe89f5981837 FAIL (2026-05-10 14:01:22 UTC) orbiter warning: context deadline exceeded Duration: 4.190008249s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..................................................X.X.......] duration: 4.004857845s trace: https://jaeger.dev.storj.io/trace/2a36fe89f5981837 FAIL (2026-05-10 14:00:27 UTC) orbiter warning: context deadline exceeded Duration: 4.004881982s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [X..................................................X.X......] duration: 4.401777522s trace: https://jaeger.dev.storj.io/trace/2a36fe89f5981837 FAIL (2026-05-10 13:59:52 UTC) orbiter warning: context deadline exceeded Duration: 4.401796669s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.X..................................................X.X.....] duration: 3.923549224s trace: https://jaeger.dev.storj.io/trace/2a36fe89f5981837 FAIL (2026-05-10 13:59:10 UTC) orbiter warning: context deadline exceeded Duration: 3.923561838s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)