Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.797476394s trace: https://jaeger.dev.storj.io/trace/9c10d1cea23ecd9 FAIL (2026-01-09 09:25:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12HWX3mC14qJg7WdCetaBtS2GcGQFhzSaqUbKrYFBHZKdyFAAgX, Piece ID: ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA) context deadline exceeded; piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A) context deadline exceeded; piecestore: (Node ID: 1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, Piece ID: 3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ) context deadline exceeded ======LOGS====== Jan 9 08:42:09.786209491 (833h13m9.657292536s) [upload=728822]: Starting upload Jan 9 08:42:09.786265998 (833h13m9.657349033s) [upload=728822]: Done starting upload Jan 9 08:42:09.786338025 (833h13m9.65742106s) [upload=728822]: Uploading segments... Jan 9 08:42:09.786470407 (833h13m9.657553443s) [upload=728822]: Got next segment. Inline: false Jan 9 08:42:09.786499243 (833h13m9.657582278s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 08:42:09.786503601 (833h13m9.657586636s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 08:42:10.028227298 (833h13m9.899310333s) [seg_pos={0 0}, upload=728822]: Begin upload segment... Jan 9 08:42:10.028276782 (833h13m9.899359817s) [seg_pos={0 0}, upload=728822]: Done begin upload segment. Jan 9 08:42:10.028298404 (833h13m9.899381439s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028312550 (833h13m9.899395585s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028351725 (833h13m9.89943477s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028742609 (833h13m9.899825664s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029058711 (833h13m9.900141756s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029428165 (833h13m9.9005112s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029762632 (833h13m9.900845667s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030105314 (833h13m9.901188379s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030547576 (833h13m9.901630621s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030977594 (833h13m9.90206063s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.031805010 (833h13m9.902888055s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.032269375 (833h13m9.90335242s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.033511369 (833h13m9.904594415s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.034667751 (833h13m9.905750796s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.035948640 (833h13m9.907031685s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.037387751 (833h13m9.908470786s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:12.640854837 (833h13m12.511937872s) [upload=728822]: Next returned nil segment Jan 9 08:42:12.640867422 (833h13m12.511950457s) [upload=728822]: Waiting for error group managing segments... Jan 9 08:42:13.046761163 (833h13m12.917844209s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.084597619 (833h13m12.955680664s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.092874369 (833h13m12.963957404s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.150407240 (833h13m13.021490275s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.159991590 (833h13m13.031074625s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.176804004 (833h13m13.047887039s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.182523755 (833h13m13.0536068s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.979222818 (833h13m13.850305863s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.006239722 (833h13m13.877322757s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.059101749 (833h13m13.930184795s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.140860029 (833h13m14.011943074s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.186445952 (833h13m14.057528987s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266335753 (833h13m14.137418788s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266359047 (833h13m14.137442092s) [seg_pos={0 0}, upload=728822]: Segment reached optimal threshold of 13 pieces. Jan 9 08:42:14.266656384 (833h13m14.137739429s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9): protocol: read tcp 188.166.225.170:56458->145.79.227.22:4510: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 9 08:42:14.266666573 (833h13m14.137749618s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266732248 (833h13m14.137815303s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266745052 (833h13m14.137828127s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266905067 (833h13m14.137988102s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266913643 (833h13m14.137996699s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266932269 (833h13m14.138015314s) [seg_pos={0 0}, upload=728822]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 08:42:14.267007192 (833h13m14.138090227s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 08:42:14.267013323 (833h13m14.138096368s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 08:42:14.561620288 (833h13m14.432703333s) [upload=728822]: Done uploading segments... Jan 9 08:42:14.561646929 (833h13m14.432729974s) [upload=728822]: Upload finished. err: <nil> Duration: 5.797507673s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 48 out of 60 is failed (threshold=6) [.XX.XXXXXXXX.XXXXXXXXX.XXXXXXXXXX.XXXX.XXXXX.XX...XXXXXX.X.X] duration: 5.001158959s trace: https://jaeger.dev.storj.io/trace/a11d5406af98eda FAIL (2026-01-09 09:24:51 UTC) orbiter warning: context deadline exceeded Duration: 5.001184226s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...X.X..................X...................................] duration: 22.340113083s trace: https://jaeger.dev.storj.io/trace/32dce7166c4df329 FAIL (2026-01-09 09:24:51 UTC) orbiter warning: context deadline exceeded Duration: 22.340139333s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.......X.X...XX.XX...X...........XXXX...............X.X.....] duration: 20.642281111s trace: https://jaeger.dev.storj.io/trace/4a77942169307775 FAIL (2026-01-09 09:24:36 UTC) orbiter warning: context deadline exceeded Duration: 20.642313522s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.90959303s trace: https://jaeger.dev.storj.io/trace/9c10d1cea23ecd9 FAIL (2026-01-09 09:24:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12HWX3mC14qJg7WdCetaBtS2GcGQFhzSaqUbKrYFBHZKdyFAAgX, Piece ID: ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA) context deadline exceeded; piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A) context deadline exceeded; piecestore: (Node ID: 1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, Piece ID: 3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ) context deadline exceeded ======LOGS====== Jan 9 08:42:09.786209491 (833h13m9.657292536s) [upload=728822]: Starting upload Jan 9 08:42:09.786265998 (833h13m9.657349033s) [upload=728822]: Done starting upload Jan 9 08:42:09.786338025 (833h13m9.65742106s) [upload=728822]: Uploading segments... Jan 9 08:42:09.786470407 (833h13m9.657553443s) [upload=728822]: Got next segment. Inline: false Jan 9 08:42:09.786499243 (833h13m9.657582278s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 08:42:09.786503601 (833h13m9.657586636s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 08:42:10.028227298 (833h13m9.899310333s) [seg_pos={0 0}, upload=728822]: Begin upload segment... Jan 9 08:42:10.028276782 (833h13m9.899359817s) [seg_pos={0 0}, upload=728822]: Done begin upload segment. Jan 9 08:42:10.028298404 (833h13m9.899381439s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028312550 (833h13m9.899395585s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028351725 (833h13m9.89943477s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028742609 (833h13m9.899825664s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029058711 (833h13m9.900141756s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029428165 (833h13m9.9005112s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029762632 (833h13m9.900845667s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030105314 (833h13m9.901188379s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030547576 (833h13m9.901630621s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030977594 (833h13m9.90206063s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.031805010 (833h13m9.902888055s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.032269375 (833h13m9.90335242s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.033511369 (833h13m9.904594415s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.034667751 (833h13m9.905750796s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.035948640 (833h13m9.907031685s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.037387751 (833h13m9.908470786s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:12.640854837 (833h13m12.511937872s) [upload=728822]: Next returned nil segment Jan 9 08:42:12.640867422 (833h13m12.511950457s) [upload=728822]: Waiting for error group managing segments... Jan 9 08:42:13.046761163 (833h13m12.917844209s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.084597619 (833h13m12.955680664s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.092874369 (833h13m12.963957404s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.150407240 (833h13m13.021490275s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.159991590 (833h13m13.031074625s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.176804004 (833h13m13.047887039s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.182523755 (833h13m13.0536068s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.979222818 (833h13m13.850305863s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.006239722 (833h13m13.877322757s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.059101749 (833h13m13.930184795s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.140860029 (833h13m14.011943074s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.186445952 (833h13m14.057528987s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266335753 (833h13m14.137418788s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266359047 (833h13m14.137442092s) [seg_pos={0 0}, upload=728822]: Segment reached optimal threshold of 13 pieces. Jan 9 08:42:14.266656384 (833h13m14.137739429s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9): protocol: read tcp 188.166.225.170:56458->145.79.227.22:4510: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 9 08:42:14.266666573 (833h13m14.137749618s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266732248 (833h13m14.137815303s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266745052 (833h13m14.137828127s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266905067 (833h13m14.137988102s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266913643 (833h13m14.137996699s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266932269 (833h13m14.138015314s) [seg_pos={0 0}, upload=728822]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 08:42:14.267007192 (833h13m14.138090227s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 08:42:14.267013323 (833h13m14.138096368s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 08:42:14.561620288 (833h13m14.432703333s) [upload=728822]: Done uploading segments... Jan 9 08:42:14.561646929 (833h13m14.432729974s) [upload=728822]: Upload finished. err: <nil> Duration: 5.909625912s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [....X.X..................X..................................] duration: 20.385325894s trace: https://jaeger.dev.storj.io/trace/32dce7166c4df329 FAIL (2026-01-09 09:23:51 UTC) orbiter warning: context deadline exceeded Duration: 20.385351372s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 48 out of 60 is failed (threshold=6) [X.XX.XXXXXXXX.XXXXXXXXX.XXXXXXXXXX.XXXX.XXXXX.XX...XXXXXX.X.] duration: 4.506492325s trace: https://jaeger.dev.storj.io/trace/7e1db7cd8fa300d9 FAIL (2026-01-09 09:23:36 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.506518574s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.807077141s trace: https://jaeger.dev.storj.io/trace/9c10d1cea23ecd9 FAIL (2026-01-09 09:23:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12HWX3mC14qJg7WdCetaBtS2GcGQFhzSaqUbKrYFBHZKdyFAAgX, Piece ID: ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA) context deadline exceeded; piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A) context deadline exceeded; piecestore: (Node ID: 1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, Piece ID: 3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ) context deadline exceeded ======LOGS====== Jan 9 08:42:09.786209491 (833h13m9.657292536s) [upload=728822]: Starting upload Jan 9 08:42:09.786265998 (833h13m9.657349033s) [upload=728822]: Done starting upload Jan 9 08:42:09.786338025 (833h13m9.65742106s) [upload=728822]: Uploading segments... Jan 9 08:42:09.786470407 (833h13m9.657553443s) [upload=728822]: Got next segment. Inline: false Jan 9 08:42:09.786499243 (833h13m9.657582278s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 08:42:09.786503601 (833h13m9.657586636s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 08:42:10.028227298 (833h13m9.899310333s) [seg_pos={0 0}, upload=728822]: Begin upload segment... Jan 9 08:42:10.028276782 (833h13m9.899359817s) [seg_pos={0 0}, upload=728822]: Done begin upload segment. Jan 9 08:42:10.028298404 (833h13m9.899381439s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028312550 (833h13m9.899395585s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028351725 (833h13m9.89943477s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028742609 (833h13m9.899825664s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029058711 (833h13m9.900141756s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029428165 (833h13m9.9005112s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029762632 (833h13m9.900845667s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030105314 (833h13m9.901188379s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030547576 (833h13m9.901630621s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030977594 (833h13m9.90206063s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.031805010 (833h13m9.902888055s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.032269375 (833h13m9.90335242s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.033511369 (833h13m9.904594415s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.034667751 (833h13m9.905750796s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.035948640 (833h13m9.907031685s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.037387751 (833h13m9.908470786s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:12.640854837 (833h13m12.511937872s) [upload=728822]: Next returned nil segment Jan 9 08:42:12.640867422 (833h13m12.511950457s) [upload=728822]: Waiting for error group managing segments... Jan 9 08:42:13.046761163 (833h13m12.917844209s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.084597619 (833h13m12.955680664s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.092874369 (833h13m12.963957404s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.150407240 (833h13m13.021490275s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.159991590 (833h13m13.031074625s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.176804004 (833h13m13.047887039s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.182523755 (833h13m13.0536068s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.979222818 (833h13m13.850305863s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.006239722 (833h13m13.877322757s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.059101749 (833h13m13.930184795s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.140860029 (833h13m14.011943074s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.186445952 (833h13m14.057528987s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266335753 (833h13m14.137418788s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266359047 (833h13m14.137442092s) [seg_pos={0 0}, upload=728822]: Segment reached optimal threshold of 13 pieces. Jan 9 08:42:14.266656384 (833h13m14.137739429s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9): protocol: read tcp 188.166.225.170:56458->145.79.227.22:4510: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 9 08:42:14.266666573 (833h13m14.137749618s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266732248 (833h13m14.137815303s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266745052 (833h13m14.137828127s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266905067 (833h13m14.137988102s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266913643 (833h13m14.137996699s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266932269 (833h13m14.138015314s) [seg_pos={0 0}, upload=728822]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 08:42:14.267007192 (833h13m14.138090227s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 08:42:14.267013323 (833h13m14.138096368s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 08:42:14.561620288 (833h13m14.432703333s) [upload=728822]: Done uploading segments... Jan 9 08:42:14.561646929 (833h13m14.432729974s) [upload=728822]: Upload finished. err: <nil> Duration: 5.807107018s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [X.......X.X...XX.XX...X...........XXXX...............X.X....] duration: 10.60216738s trace: https://jaeger.dev.storj.io/trace/4a77942169307775 FAIL (2026-01-09 09:23:24 UTC) orbiter warning: context deadline exceeded Duration: 10.602210353s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.....X.X..................X.................................] duration: 17.986438311s trace: https://jaeger.dev.storj.io/trace/32dce7166c4df329 FAIL (2026-01-09 09:23:14 UTC) orbiter warning: context deadline exceeded Duration: 17.986468339s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.877938463s trace: https://jaeger.dev.storj.io/trace/9c10d1cea23ecd9 FAIL (2026-01-09 09:23:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12HWX3mC14qJg7WdCetaBtS2GcGQFhzSaqUbKrYFBHZKdyFAAgX, Piece ID: ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA) context deadline exceeded; piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A) context deadline exceeded; piecestore: (Node ID: 1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, Piece ID: 3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ) context deadline exceeded ======LOGS====== Jan 9 08:42:09.786209491 (833h13m9.657292536s) [upload=728822]: Starting upload Jan 9 08:42:09.786265998 (833h13m9.657349033s) [upload=728822]: Done starting upload Jan 9 08:42:09.786338025 (833h13m9.65742106s) [upload=728822]: Uploading segments... Jan 9 08:42:09.786470407 (833h13m9.657553443s) [upload=728822]: Got next segment. Inline: false Jan 9 08:42:09.786499243 (833h13m9.657582278s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 08:42:09.786503601 (833h13m9.657586636s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 08:42:10.028227298 (833h13m9.899310333s) [seg_pos={0 0}, upload=728822]: Begin upload segment... Jan 9 08:42:10.028276782 (833h13m9.899359817s) [seg_pos={0 0}, upload=728822]: Done begin upload segment. Jan 9 08:42:10.028298404 (833h13m9.899381439s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028312550 (833h13m9.899395585s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028351725 (833h13m9.89943477s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028742609 (833h13m9.899825664s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029058711 (833h13m9.900141756s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029428165 (833h13m9.9005112s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029762632 (833h13m9.900845667s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030105314 (833h13m9.901188379s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030547576 (833h13m9.901630621s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030977594 (833h13m9.90206063s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.031805010 (833h13m9.902888055s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.032269375 (833h13m9.90335242s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.033511369 (833h13m9.904594415s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.034667751 (833h13m9.905750796s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.035948640 (833h13m9.907031685s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.037387751 (833h13m9.908470786s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:12.640854837 (833h13m12.511937872s) [upload=728822]: Next returned nil segment Jan 9 08:42:12.640867422 (833h13m12.511950457s) [upload=728822]: Waiting for error group managing segments... Jan 9 08:42:13.046761163 (833h13m12.917844209s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.084597619 (833h13m12.955680664s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.092874369 (833h13m12.963957404s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.150407240 (833h13m13.021490275s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.159991590 (833h13m13.031074625s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.176804004 (833h13m13.047887039s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.182523755 (833h13m13.0536068s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.979222818 (833h13m13.850305863s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.006239722 (833h13m13.877322757s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.059101749 (833h13m13.930184795s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.140860029 (833h13m14.011943074s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.186445952 (833h13m14.057528987s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266335753 (833h13m14.137418788s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266359047 (833h13m14.137442092s) [seg_pos={0 0}, upload=728822]: Segment reached optimal threshold of 13 pieces. Jan 9 08:42:14.266656384 (833h13m14.137739429s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9): protocol: read tcp 188.166.225.170:56458->145.79.227.22:4510: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 9 08:42:14.266666573 (833h13m14.137749618s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266732248 (833h13m14.137815303s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266745052 (833h13m14.137828127s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266905067 (833h13m14.137988102s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266913643 (833h13m14.137996699s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266932269 (833h13m14.138015314s) [seg_pos={0 0}, upload=728822]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 08:42:14.267007192 (833h13m14.138090227s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 08:42:14.267013323 (833h13m14.138096368s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 08:42:14.561620288 (833h13m14.432703333s) [upload=728822]: Done uploading segments... Jan 9 08:42:14.561646929 (833h13m14.432729974s) [upload=728822]: Upload finished. err: <nil> Duration: 5.877966156s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.793147331s trace: https://jaeger.dev.storj.io/trace/9c10d1cea23ecd9 FAIL (2026-01-09 09:22:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12HWX3mC14qJg7WdCetaBtS2GcGQFhzSaqUbKrYFBHZKdyFAAgX, Piece ID: ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA) context deadline exceeded; piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A) context deadline exceeded; piecestore: (Node ID: 1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, Piece ID: 3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ) context deadline exceeded ======LOGS====== Jan 9 08:42:09.786209491 (833h13m9.657292536s) [upload=728822]: Starting upload Jan 9 08:42:09.786265998 (833h13m9.657349033s) [upload=728822]: Done starting upload Jan 9 08:42:09.786338025 (833h13m9.65742106s) [upload=728822]: Uploading segments... Jan 9 08:42:09.786470407 (833h13m9.657553443s) [upload=728822]: Got next segment. Inline: false Jan 9 08:42:09.786499243 (833h13m9.657582278s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 08:42:09.786503601 (833h13m9.657586636s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 08:42:10.028227298 (833h13m9.899310333s) [seg_pos={0 0}, upload=728822]: Begin upload segment... Jan 9 08:42:10.028276782 (833h13m9.899359817s) [seg_pos={0 0}, upload=728822]: Done begin upload segment. Jan 9 08:42:10.028298404 (833h13m9.899381439s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028312550 (833h13m9.899395585s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028351725 (833h13m9.89943477s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028742609 (833h13m9.899825664s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029058711 (833h13m9.900141756s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029428165 (833h13m9.9005112s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029762632 (833h13m9.900845667s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030105314 (833h13m9.901188379s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030547576 (833h13m9.901630621s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030977594 (833h13m9.90206063s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.031805010 (833h13m9.902888055s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.032269375 (833h13m9.90335242s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.033511369 (833h13m9.904594415s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.034667751 (833h13m9.905750796s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.035948640 (833h13m9.907031685s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.037387751 (833h13m9.908470786s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:12.640854837 (833h13m12.511937872s) [upload=728822]: Next returned nil segment Jan 9 08:42:12.640867422 (833h13m12.511950457s) [upload=728822]: Waiting for error group managing segments... Jan 9 08:42:13.046761163 (833h13m12.917844209s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.084597619 (833h13m12.955680664s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.092874369 (833h13m12.963957404s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.150407240 (833h13m13.021490275s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.159991590 (833h13m13.031074625s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.176804004 (833h13m13.047887039s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.182523755 (833h13m13.0536068s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.979222818 (833h13m13.850305863s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.006239722 (833h13m13.877322757s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.059101749 (833h13m13.930184795s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.140860029 (833h13m14.011943074s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.186445952 (833h13m14.057528987s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266335753 (833h13m14.137418788s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266359047 (833h13m14.137442092s) [seg_pos={0 0}, upload=728822]: Segment reached optimal threshold of 13 pieces. Jan 9 08:42:14.266656384 (833h13m14.137739429s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9): protocol: read tcp 188.166.225.170:56458->145.79.227.22:4510: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 9 08:42:14.266666573 (833h13m14.137749618s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266732248 (833h13m14.137815303s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266745052 (833h13m14.137828127s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266905067 (833h13m14.137988102s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266913643 (833h13m14.137996699s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266932269 (833h13m14.138015314s) [seg_pos={0 0}, upload=728822]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 08:42:14.267007192 (833h13m14.138090227s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 08:42:14.267013323 (833h13m14.138096368s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 08:42:14.561620288 (833h13m14.432703333s) [upload=728822]: Done uploading segments... Jan 9 08:42:14.561646929 (833h13m14.432729974s) [upload=728822]: Upload finished. err: <nil> Duration: 5.793175255s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 48 out of 60 is failed (threshold=6) [.X.XX.XXXXXXXX.XXXXXXXXX.XXXXXXXXXX.XXXX.XXXXX.XX...XXXXXX.X] duration: 5.001718756s trace: https://jaeger.dev.storj.io/trace/7e1db7cd8fa300d9 FAIL (2026-01-09 09:22:38 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001742061s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [.X.......X.X...XX.XX...X...........XXXX...............X.X...] duration: 22.183450407s trace: https://jaeger.dev.storj.io/trace/4a77942169307775 FAIL (2026-01-09 09:22:34 UTC) orbiter warning: context deadline exceeded Duration: 22.183480134s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [..X.......X.X...XX.XX...X...........XXXX...............X.X..] duration: 21.979634474s trace: https://jaeger.dev.storj.io/trace/4a77942169307775 FAIL (2026-01-09 09:21:59 UTC) orbiter warning: context deadline exceeded Duration: 21.97966333s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.848125707s trace: https://jaeger.dev.storj.io/trace/9c10d1cea23ecd9 FAIL (2026-01-09 09:21:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12HWX3mC14qJg7WdCetaBtS2GcGQFhzSaqUbKrYFBHZKdyFAAgX, Piece ID: ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA) context deadline exceeded; piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A) context deadline exceeded; piecestore: (Node ID: 1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, Piece ID: 3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ) context deadline exceeded ======LOGS====== Jan 9 08:42:09.786209491 (833h13m9.657292536s) [upload=728822]: Starting upload Jan 9 08:42:09.786265998 (833h13m9.657349033s) [upload=728822]: Done starting upload Jan 9 08:42:09.786338025 (833h13m9.65742106s) [upload=728822]: Uploading segments... Jan 9 08:42:09.786470407 (833h13m9.657553443s) [upload=728822]: Got next segment. Inline: false Jan 9 08:42:09.786499243 (833h13m9.657582278s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 08:42:09.786503601 (833h13m9.657586636s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 08:42:10.028227298 (833h13m9.899310333s) [seg_pos={0 0}, upload=728822]: Begin upload segment... Jan 9 08:42:10.028276782 (833h13m9.899359817s) [seg_pos={0 0}, upload=728822]: Done begin upload segment. Jan 9 08:42:10.028298404 (833h13m9.899381439s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028312550 (833h13m9.899395585s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028351725 (833h13m9.89943477s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028742609 (833h13m9.899825664s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029058711 (833h13m9.900141756s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029428165 (833h13m9.9005112s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029762632 (833h13m9.900845667s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030105314 (833h13m9.901188379s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030547576 (833h13m9.901630621s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030977594 (833h13m9.90206063s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.031805010 (833h13m9.902888055s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.032269375 (833h13m9.90335242s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.033511369 (833h13m9.904594415s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.034667751 (833h13m9.905750796s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.035948640 (833h13m9.907031685s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.037387751 (833h13m9.908470786s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:12.640854837 (833h13m12.511937872s) [upload=728822]: Next returned nil segment Jan 9 08:42:12.640867422 (833h13m12.511950457s) [upload=728822]: Waiting for error group managing segments... Jan 9 08:42:13.046761163 (833h13m12.917844209s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.084597619 (833h13m12.955680664s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.092874369 (833h13m12.963957404s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.150407240 (833h13m13.021490275s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.159991590 (833h13m13.031074625s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.176804004 (833h13m13.047887039s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.182523755 (833h13m13.0536068s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.979222818 (833h13m13.850305863s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.006239722 (833h13m13.877322757s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.059101749 (833h13m13.930184795s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.140860029 (833h13m14.011943074s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.186445952 (833h13m14.057528987s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266335753 (833h13m14.137418788s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266359047 (833h13m14.137442092s) [seg_pos={0 0}, upload=728822]: Segment reached optimal threshold of 13 pieces. Jan 9 08:42:14.266656384 (833h13m14.137739429s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9): protocol: read tcp 188.166.225.170:56458->145.79.227.22:4510: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 9 08:42:14.266666573 (833h13m14.137749618s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266732248 (833h13m14.137815303s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266745052 (833h13m14.137828127s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266905067 (833h13m14.137988102s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266913643 (833h13m14.137996699s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266932269 (833h13m14.138015314s) [seg_pos={0 0}, upload=728822]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 08:42:14.267007192 (833h13m14.138090227s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 08:42:14.267013323 (833h13m14.138096368s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 08:42:14.561620288 (833h13m14.432703333s) [upload=728822]: Done uploading segments... Jan 9 08:42:14.561646929 (833h13m14.432729974s) [upload=728822]: Upload finished. err: <nil> Duration: 5.84815844s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [......X.X..................X................................] duration: 7.187163564s trace: https://jaeger.dev.storj.io/trace/32dce7166c4df329 FAIL (2026-01-09 09:21:51 UTC) orbiter warning: context deadline exceeded Duration: 7.187189043s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.......X.X..................X...............................] duration: 18.100939069s trace: https://jaeger.dev.storj.io/trace/32dce7166c4df329 FAIL (2026-01-09 09:21:35 UTC) orbiter warning: context deadline exceeded Duration: 18.10097113s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 47 out of 60 is failed (threshold=6) [..X.XX.XXXXXXXX.XXXXXXXXX.XXXXXXXXXX.XXXX.XXXXX.XX...XXXXXX.] duration: 3.898362322s trace: https://jaeger.dev.storj.io/trace/145044b0280f00fa FAIL (2026-01-09 09:21:34 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.89838778s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [...X.......X.X...XX.XX...X...........XXXX...............X.X.] duration: 28.14206695s trace: https://jaeger.dev.storj.io/trace/4a77942169307775 FAIL (2026-01-09 09:21:26 UTC) orbiter warning: context deadline exceeded Duration: 28.142092588s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 48 out of 60 is failed (threshold=6) [X..X.XX.XXXXXXXX.XXXXXXXXX.XXXXXXXXXX.XXXX.XXXXX.XX...XXXXXX] duration: 5.001780731s trace: https://jaeger.dev.storj.io/trace/145044b0280f00fa FAIL (2026-01-09 09:21:17 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001805808s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [........X.X..................X..............................] duration: 18.247503399s trace: https://jaeger.dev.storj.io/trace/32dce7166c4df329 FAIL (2026-01-09 09:21:03 UTC) orbiter warning: context deadline exceeded Duration: 18.247532083s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 5.783731014s trace: https://jaeger.dev.storj.io/trace/9c10d1cea23ecd9 FAIL (2026-01-09 09:20:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12HWX3mC14qJg7WdCetaBtS2GcGQFhzSaqUbKrYFBHZKdyFAAgX, Piece ID: ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA) context deadline exceeded; piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A) context deadline exceeded; piecestore: (Node ID: 1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, Piece ID: 3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ) context deadline exceeded ======LOGS====== Jan 9 08:42:09.786209491 (833h13m9.657292536s) [upload=728822]: Starting upload Jan 9 08:42:09.786265998 (833h13m9.657349033s) [upload=728822]: Done starting upload Jan 9 08:42:09.786338025 (833h13m9.65742106s) [upload=728822]: Uploading segments... Jan 9 08:42:09.786470407 (833h13m9.657553443s) [upload=728822]: Got next segment. Inline: false Jan 9 08:42:09.786499243 (833h13m9.657582278s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 08:42:09.786503601 (833h13m9.657586636s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 08:42:10.028227298 (833h13m9.899310333s) [seg_pos={0 0}, upload=728822]: Begin upload segment... Jan 9 08:42:10.028276782 (833h13m9.899359817s) [seg_pos={0 0}, upload=728822]: Done begin upload segment. Jan 9 08:42:10.028298404 (833h13m9.899381439s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028312550 (833h13m9.899395585s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028351725 (833h13m9.89943477s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028742609 (833h13m9.899825664s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029058711 (833h13m9.900141756s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029428165 (833h13m9.9005112s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029762632 (833h13m9.900845667s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030105314 (833h13m9.901188379s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030547576 (833h13m9.901630621s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030977594 (833h13m9.90206063s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.031805010 (833h13m9.902888055s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.032269375 (833h13m9.90335242s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.033511369 (833h13m9.904594415s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.034667751 (833h13m9.905750796s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.035948640 (833h13m9.907031685s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.037387751 (833h13m9.908470786s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:12.640854837 (833h13m12.511937872s) [upload=728822]: Next returned nil segment Jan 9 08:42:12.640867422 (833h13m12.511950457s) [upload=728822]: Waiting for error group managing segments... Jan 9 08:42:13.046761163 (833h13m12.917844209s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.084597619 (833h13m12.955680664s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.092874369 (833h13m12.963957404s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.150407240 (833h13m13.021490275s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.159991590 (833h13m13.031074625s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.176804004 (833h13m13.047887039s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.182523755 (833h13m13.0536068s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.979222818 (833h13m13.850305863s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.006239722 (833h13m13.877322757s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.059101749 (833h13m13.930184795s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.140860029 (833h13m14.011943074s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.186445952 (833h13m14.057528987s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266335753 (833h13m14.137418788s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266359047 (833h13m14.137442092s) [seg_pos={0 0}, upload=728822]: Segment reached optimal threshold of 13 pieces. Jan 9 08:42:14.266656384 (833h13m14.137739429s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9): protocol: read tcp 188.166.225.170:56458->145.79.227.22:4510: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 9 08:42:14.266666573 (833h13m14.137749618s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266732248 (833h13m14.137815303s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266745052 (833h13m14.137828127s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266905067 (833h13m14.137988102s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266913643 (833h13m14.137996699s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266932269 (833h13m14.138015314s) [seg_pos={0 0}, upload=728822]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 08:42:14.267007192 (833h13m14.138090227s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 08:42:14.267013323 (833h13m14.138096368s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 08:42:14.561620288 (833h13m14.432703333s) [upload=728822]: Done uploading segments... Jan 9 08:42:14.561646929 (833h13m14.432729974s) [upload=728822]: Upload finished. err: <nil> Duration: 5.78376033s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 48 out of 60 is failed (threshold=6) [XX..X.XX.XXXXXXXX.XXXXXXXXX.XXXXXXXXXX.XXXX.XXXXX.XX...XXXXX] duration: 5.000789026s trace: https://jaeger.dev.storj.io/trace/3a4c67164b66dd5a FAIL (2026-01-09 09:20:49 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000814194s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [....X.......X.X...XX.XX...X...........XXXX...............X.X] duration: 30.001341258s trace: https://jaeger.dev.storj.io/trace/4a77942169307775 FAIL (2026-01-09 09:20:31 UTC) orbiter warning: context deadline exceeded Duration: 30.00135222s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.........X.X..................X.............................] duration: 8.185143672s trace: https://jaeger.dev.storj.io/trace/32dce7166c4df329 FAIL (2026-01-09 09:20:08 UTC) orbiter warning: context deadline exceeded Duration: 8.185171874s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.789638368s trace: https://jaeger.dev.storj.io/trace/9c10d1cea23ecd9 FAIL (2026-01-09 09:20:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12HWX3mC14qJg7WdCetaBtS2GcGQFhzSaqUbKrYFBHZKdyFAAgX, Piece ID: ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA) context deadline exceeded; piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A) context deadline exceeded; piecestore: (Node ID: 1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, Piece ID: 3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ) context deadline exceeded ======LOGS====== Jan 9 08:42:09.786209491 (833h13m9.657292536s) [upload=728822]: Starting upload Jan 9 08:42:09.786265998 (833h13m9.657349033s) [upload=728822]: Done starting upload Jan 9 08:42:09.786338025 (833h13m9.65742106s) [upload=728822]: Uploading segments... Jan 9 08:42:09.786470407 (833h13m9.657553443s) [upload=728822]: Got next segment. Inline: false Jan 9 08:42:09.786499243 (833h13m9.657582278s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 08:42:09.786503601 (833h13m9.657586636s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 08:42:10.028227298 (833h13m9.899310333s) [seg_pos={0 0}, upload=728822]: Begin upload segment... Jan 9 08:42:10.028276782 (833h13m9.899359817s) [seg_pos={0 0}, upload=728822]: Done begin upload segment. Jan 9 08:42:10.028298404 (833h13m9.899381439s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028312550 (833h13m9.899395585s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028351725 (833h13m9.89943477s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028742609 (833h13m9.899825664s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029058711 (833h13m9.900141756s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029428165 (833h13m9.9005112s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029762632 (833h13m9.900845667s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030105314 (833h13m9.901188379s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030547576 (833h13m9.901630621s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030977594 (833h13m9.90206063s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.031805010 (833h13m9.902888055s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.032269375 (833h13m9.90335242s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.033511369 (833h13m9.904594415s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.034667751 (833h13m9.905750796s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.035948640 (833h13m9.907031685s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.037387751 (833h13m9.908470786s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:12.640854837 (833h13m12.511937872s) [upload=728822]: Next returned nil segment Jan 9 08:42:12.640867422 (833h13m12.511950457s) [upload=728822]: Waiting for error group managing segments... Jan 9 08:42:13.046761163 (833h13m12.917844209s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.084597619 (833h13m12.955680664s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.092874369 (833h13m12.963957404s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.150407240 (833h13m13.021490275s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.159991590 (833h13m13.031074625s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.176804004 (833h13m13.047887039s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.182523755 (833h13m13.0536068s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.979222818 (833h13m13.850305863s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.006239722 (833h13m13.877322757s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.059101749 (833h13m13.930184795s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.140860029 (833h13m14.011943074s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.186445952 (833h13m14.057528987s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266335753 (833h13m14.137418788s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266359047 (833h13m14.137442092s) [seg_pos={0 0}, upload=728822]: Segment reached optimal threshold of 13 pieces. Jan 9 08:42:14.266656384 (833h13m14.137739429s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9): protocol: read tcp 188.166.225.170:56458->145.79.227.22:4510: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 9 08:42:14.266666573 (833h13m14.137749618s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266732248 (833h13m14.137815303s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266745052 (833h13m14.137828127s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266905067 (833h13m14.137988102s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266913643 (833h13m14.137996699s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266932269 (833h13m14.138015314s) [seg_pos={0 0}, upload=728822]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 08:42:14.267007192 (833h13m14.138090227s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 08:42:14.267013323 (833h13m14.138096368s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 08:42:14.561620288 (833h13m14.432703333s) [upload=728822]: Done uploading segments... Jan 9 08:42:14.561646929 (833h13m14.432729974s) [upload=728822]: Upload finished. err: <nil> Duration: 5.78966575s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.....X.......X.X...XX.XX...X...........XXXX...............X.] duration: 28.041426328s trace: https://jaeger.dev.storj.io/trace/193d17517e84e836 FAIL (2026-01-09 09:20:00 UTC) orbiter warning: context deadline exceeded Duration: 28.041451057s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 48 out of 60 is failed (threshold=6) [XXX..X.XX.XXXXXXXX.XXXXXXXXX.XXXXXXXXXX.XXXX.XXXXX.XX...XXXX] duration: 5.00115996s trace: https://jaeger.dev.storj.io/trace/5ce8e72a52fd7737 FAIL (2026-01-09 09:19:41 UTC) orbiter warning: context deadline exceeded Duration: 5.001169718s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.802094286s trace: https://jaeger.dev.storj.io/trace/9c10d1cea23ecd9 FAIL (2026-01-09 09:19:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12HWX3mC14qJg7WdCetaBtS2GcGQFhzSaqUbKrYFBHZKdyFAAgX, Piece ID: ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA) context deadline exceeded; piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A) context deadline exceeded; piecestore: (Node ID: 1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, Piece ID: 3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ) context deadline exceeded ======LOGS====== Jan 9 08:42:09.786209491 (833h13m9.657292536s) [upload=728822]: Starting upload Jan 9 08:42:09.786265998 (833h13m9.657349033s) [upload=728822]: Done starting upload Jan 9 08:42:09.786338025 (833h13m9.65742106s) [upload=728822]: Uploading segments... Jan 9 08:42:09.786470407 (833h13m9.657553443s) [upload=728822]: Got next segment. Inline: false Jan 9 08:42:09.786499243 (833h13m9.657582278s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 08:42:09.786503601 (833h13m9.657586636s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 08:42:10.028227298 (833h13m9.899310333s) [seg_pos={0 0}, upload=728822]: Begin upload segment... Jan 9 08:42:10.028276782 (833h13m9.899359817s) [seg_pos={0 0}, upload=728822]: Done begin upload segment. Jan 9 08:42:10.028298404 (833h13m9.899381439s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028312550 (833h13m9.899395585s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028351725 (833h13m9.89943477s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028742609 (833h13m9.899825664s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029058711 (833h13m9.900141756s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029428165 (833h13m9.9005112s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029762632 (833h13m9.900845667s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030105314 (833h13m9.901188379s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030547576 (833h13m9.901630621s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030977594 (833h13m9.90206063s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.031805010 (833h13m9.902888055s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.032269375 (833h13m9.90335242s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.033511369 (833h13m9.904594415s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.034667751 (833h13m9.905750796s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.035948640 (833h13m9.907031685s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.037387751 (833h13m9.908470786s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:12.640854837 (833h13m12.511937872s) [upload=728822]: Next returned nil segment Jan 9 08:42:12.640867422 (833h13m12.511950457s) [upload=728822]: Waiting for error group managing segments... Jan 9 08:42:13.046761163 (833h13m12.917844209s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.084597619 (833h13m12.955680664s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.092874369 (833h13m12.963957404s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.150407240 (833h13m13.021490275s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.159991590 (833h13m13.031074625s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.176804004 (833h13m13.047887039s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.182523755 (833h13m13.0536068s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.979222818 (833h13m13.850305863s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.006239722 (833h13m13.877322757s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.059101749 (833h13m13.930184795s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.140860029 (833h13m14.011943074s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.186445952 (833h13m14.057528987s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266335753 (833h13m14.137418788s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266359047 (833h13m14.137442092s) [seg_pos={0 0}, upload=728822]: Segment reached optimal threshold of 13 pieces. Jan 9 08:42:14.266656384 (833h13m14.137739429s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9): protocol: read tcp 188.166.225.170:56458->145.79.227.22:4510: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 9 08:42:14.266666573 (833h13m14.137749618s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266732248 (833h13m14.137815303s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266745052 (833h13m14.137828127s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266905067 (833h13m14.137988102s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266913643 (833h13m14.137996699s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266932269 (833h13m14.138015314s) [seg_pos={0 0}, upload=728822]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 08:42:14.267007192 (833h13m14.138090227s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 08:42:14.267013323 (833h13m14.138096368s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 08:42:14.561620288 (833h13m14.432703333s) [upload=728822]: Done uploading segments... Jan 9 08:42:14.561646929 (833h13m14.432729974s) [upload=728822]: Upload finished. err: <nil> Duration: 5.80212299s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [......X.......X.X...XX.XX...X...........XXXX...............X] duration: 30.000786618s trace: https://jaeger.dev.storj.io/trace/193d17517e84e836 FAIL (2026-01-09 09:19:17 UTC) orbiter warning: context deadline exceeded Duration: 30.000797728s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [..........X.X..................X............................] duration: 7.2815896s trace: https://jaeger.dev.storj.io/trace/32dce7166c4df329 FAIL (2026-01-09 09:18:55 UTC) orbiter warning: context deadline exceeded Duration: 7.281619156s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 5.900053152s trace: https://jaeger.dev.storj.io/trace/9c10d1cea23ecd9 FAIL (2026-01-09 09:18:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12HWX3mC14qJg7WdCetaBtS2GcGQFhzSaqUbKrYFBHZKdyFAAgX, Piece ID: ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA) context deadline exceeded; piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A) context deadline exceeded; piecestore: (Node ID: 1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, Piece ID: 3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ) context deadline exceeded ======LOGS====== Jan 9 08:42:09.786209491 (833h13m9.657292536s) [upload=728822]: Starting upload Jan 9 08:42:09.786265998 (833h13m9.657349033s) [upload=728822]: Done starting upload Jan 9 08:42:09.786338025 (833h13m9.65742106s) [upload=728822]: Uploading segments... Jan 9 08:42:09.786470407 (833h13m9.657553443s) [upload=728822]: Got next segment. Inline: false Jan 9 08:42:09.786499243 (833h13m9.657582278s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 08:42:09.786503601 (833h13m9.657586636s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 08:42:10.028227298 (833h13m9.899310333s) [seg_pos={0 0}, upload=728822]: Begin upload segment... Jan 9 08:42:10.028276782 (833h13m9.899359817s) [seg_pos={0 0}, upload=728822]: Done begin upload segment. Jan 9 08:42:10.028298404 (833h13m9.899381439s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028312550 (833h13m9.899395585s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028351725 (833h13m9.89943477s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028742609 (833h13m9.899825664s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029058711 (833h13m9.900141756s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029428165 (833h13m9.9005112s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029762632 (833h13m9.900845667s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030105314 (833h13m9.901188379s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030547576 (833h13m9.901630621s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030977594 (833h13m9.90206063s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.031805010 (833h13m9.902888055s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.032269375 (833h13m9.90335242s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.033511369 (833h13m9.904594415s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.034667751 (833h13m9.905750796s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.035948640 (833h13m9.907031685s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.037387751 (833h13m9.908470786s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:12.640854837 (833h13m12.511937872s) [upload=728822]: Next returned nil segment Jan 9 08:42:12.640867422 (833h13m12.511950457s) [upload=728822]: Waiting for error group managing segments... Jan 9 08:42:13.046761163 (833h13m12.917844209s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.084597619 (833h13m12.955680664s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.092874369 (833h13m12.963957404s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.150407240 (833h13m13.021490275s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.159991590 (833h13m13.031074625s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.176804004 (833h13m13.047887039s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.182523755 (833h13m13.0536068s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.979222818 (833h13m13.850305863s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.006239722 (833h13m13.877322757s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.059101749 (833h13m13.930184795s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.140860029 (833h13m14.011943074s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.186445952 (833h13m14.057528987s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266335753 (833h13m14.137418788s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266359047 (833h13m14.137442092s) [seg_pos={0 0}, upload=728822]: Segment reached optimal threshold of 13 pieces. Jan 9 08:42:14.266656384 (833h13m14.137739429s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9): protocol: read tcp 188.166.225.170:56458->145.79.227.22:4510: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 9 08:42:14.266666573 (833h13m14.137749618s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266732248 (833h13m14.137815303s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266745052 (833h13m14.137828127s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266905067 (833h13m14.137988102s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266913643 (833h13m14.137996699s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266932269 (833h13m14.138015314s) [seg_pos={0 0}, upload=728822]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 08:42:14.267007192 (833h13m14.138090227s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 08:42:14.267013323 (833h13m14.138096368s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 08:42:14.561620288 (833h13m14.432703333s) [upload=728822]: Done uploading segments... Jan 9 08:42:14.561646929 (833h13m14.432729974s) [upload=728822]: Upload finished. err: <nil> Duration: 5.900080724s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 48 out of 60 is failed (threshold=6) [XXXX..X.XX.XXXXXXXX.XXXXXXXXX.XXXXXXXXXX.XXXX.XXXXX.XX...XXX] duration: 5.000593761s trace: https://jaeger.dev.storj.io/trace/69065ad7198c6456 FAIL (2026-01-09 09:18:46 UTC) orbiter warning: context deadline exceeded Duration: 5.000621854s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.768036966s trace: https://jaeger.dev.storj.io/trace/9c10d1cea23ecd9 FAIL (2026-01-09 09:18:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12HWX3mC14qJg7WdCetaBtS2GcGQFhzSaqUbKrYFBHZKdyFAAgX, Piece ID: ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA) context deadline exceeded; piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A) context deadline exceeded; piecestore: (Node ID: 1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, Piece ID: 3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ) context deadline exceeded ======LOGS====== Jan 9 08:42:09.786209491 (833h13m9.657292536s) [upload=728822]: Starting upload Jan 9 08:42:09.786265998 (833h13m9.657349033s) [upload=728822]: Done starting upload Jan 9 08:42:09.786338025 (833h13m9.65742106s) [upload=728822]: Uploading segments... Jan 9 08:42:09.786470407 (833h13m9.657553443s) [upload=728822]: Got next segment. Inline: false Jan 9 08:42:09.786499243 (833h13m9.657582278s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 08:42:09.786503601 (833h13m9.657586636s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 08:42:10.028227298 (833h13m9.899310333s) [seg_pos={0 0}, upload=728822]: Begin upload segment... Jan 9 08:42:10.028276782 (833h13m9.899359817s) [seg_pos={0 0}, upload=728822]: Done begin upload segment. Jan 9 08:42:10.028298404 (833h13m9.899381439s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028312550 (833h13m9.899395585s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028351725 (833h13m9.89943477s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028742609 (833h13m9.899825664s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029058711 (833h13m9.900141756s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029428165 (833h13m9.9005112s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029762632 (833h13m9.900845667s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030105314 (833h13m9.901188379s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030547576 (833h13m9.901630621s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030977594 (833h13m9.90206063s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.031805010 (833h13m9.902888055s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.032269375 (833h13m9.90335242s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.033511369 (833h13m9.904594415s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.034667751 (833h13m9.905750796s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.035948640 (833h13m9.907031685s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.037387751 (833h13m9.908470786s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:12.640854837 (833h13m12.511937872s) [upload=728822]: Next returned nil segment Jan 9 08:42:12.640867422 (833h13m12.511950457s) [upload=728822]: Waiting for error group managing segments... Jan 9 08:42:13.046761163 (833h13m12.917844209s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.084597619 (833h13m12.955680664s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.092874369 (833h13m12.963957404s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.150407240 (833h13m13.021490275s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.159991590 (833h13m13.031074625s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.176804004 (833h13m13.047887039s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.182523755 (833h13m13.0536068s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.979222818 (833h13m13.850305863s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.006239722 (833h13m13.877322757s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.059101749 (833h13m13.930184795s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.140860029 (833h13m14.011943074s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.186445952 (833h13m14.057528987s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266335753 (833h13m14.137418788s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266359047 (833h13m14.137442092s) [seg_pos={0 0}, upload=728822]: Segment reached optimal threshold of 13 pieces. Jan 9 08:42:14.266656384 (833h13m14.137739429s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9): protocol: read tcp 188.166.225.170:56458->145.79.227.22:4510: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 9 08:42:14.266666573 (833h13m14.137749618s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266732248 (833h13m14.137815303s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266745052 (833h13m14.137828127s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266905067 (833h13m14.137988102s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266913643 (833h13m14.137996699s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266932269 (833h13m14.138015314s) [seg_pos={0 0}, upload=728822]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 08:42:14.267007192 (833h13m14.138090227s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 08:42:14.267013323 (833h13m14.138096368s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 08:42:14.561620288 (833h13m14.432703333s) [upload=728822]: Done uploading segments... Jan 9 08:42:14.561646929 (833h13m14.432729974s) [upload=728822]: Upload finished. err: <nil> Duration: 5.768066182s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 47 out of 60 is failed (threshold=6) [.XXXX..X.XX.XXXXXXXX.XXXXXXXXX.XXXXXXXXXX.XXXX.XXXXX.XX...XX] duration: 5.001522695s trace: https://jaeger.dev.storj.io/trace/6930550ce48d83b2 FAIL (2026-01-09 09:18:21 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001545357s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...........X.X..................X...........................] duration: 19.200735534s trace: https://jaeger.dev.storj.io/trace/32dce7166c4df329 FAIL (2026-01-09 09:18:18 UTC) orbiter warning: context deadline exceeded Duration: 19.200761714s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.......X.......X.X...XX.XX...X...........XXXX...............] duration: 21.751837391s trace: https://jaeger.dev.storj.io/trace/41529e7a38992b5b FAIL (2026-01-09 09:18:09 UTC) orbiter warning: context deadline exceeded Duration: 21.751867168s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 5.911063046s trace: https://jaeger.dev.storj.io/trace/9c10d1cea23ecd9 FAIL (2026-01-09 09:17:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12HWX3mC14qJg7WdCetaBtS2GcGQFhzSaqUbKrYFBHZKdyFAAgX, Piece ID: ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA) context deadline exceeded; piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A) context deadline exceeded; piecestore: (Node ID: 1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, Piece ID: 3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ) context deadline exceeded ======LOGS====== Jan 9 08:42:09.786209491 (833h13m9.657292536s) [upload=728822]: Starting upload Jan 9 08:42:09.786265998 (833h13m9.657349033s) [upload=728822]: Done starting upload Jan 9 08:42:09.786338025 (833h13m9.65742106s) [upload=728822]: Uploading segments... Jan 9 08:42:09.786470407 (833h13m9.657553443s) [upload=728822]: Got next segment. Inline: false Jan 9 08:42:09.786499243 (833h13m9.657582278s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 08:42:09.786503601 (833h13m9.657586636s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 08:42:10.028227298 (833h13m9.899310333s) [seg_pos={0 0}, upload=728822]: Begin upload segment... Jan 9 08:42:10.028276782 (833h13m9.899359817s) [seg_pos={0 0}, upload=728822]: Done begin upload segment. Jan 9 08:42:10.028298404 (833h13m9.899381439s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028312550 (833h13m9.899395585s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028351725 (833h13m9.89943477s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028742609 (833h13m9.899825664s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029058711 (833h13m9.900141756s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029428165 (833h13m9.9005112s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029762632 (833h13m9.900845667s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030105314 (833h13m9.901188379s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030547576 (833h13m9.901630621s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030977594 (833h13m9.90206063s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.031805010 (833h13m9.902888055s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.032269375 (833h13m9.90335242s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.033511369 (833h13m9.904594415s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.034667751 (833h13m9.905750796s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.035948640 (833h13m9.907031685s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.037387751 (833h13m9.908470786s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:12.640854837 (833h13m12.511937872s) [upload=728822]: Next returned nil segment Jan 9 08:42:12.640867422 (833h13m12.511950457s) [upload=728822]: Waiting for error group managing segments... Jan 9 08:42:13.046761163 (833h13m12.917844209s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.084597619 (833h13m12.955680664s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.092874369 (833h13m12.963957404s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.150407240 (833h13m13.021490275s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.159991590 (833h13m13.031074625s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.176804004 (833h13m13.047887039s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.182523755 (833h13m13.0536068s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.979222818 (833h13m13.850305863s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.006239722 (833h13m13.877322757s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.059101749 (833h13m13.930184795s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.140860029 (833h13m14.011943074s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.186445952 (833h13m14.057528987s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266335753 (833h13m14.137418788s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266359047 (833h13m14.137442092s) [seg_pos={0 0}, upload=728822]: Segment reached optimal threshold of 13 pieces. Jan 9 08:42:14.266656384 (833h13m14.137739429s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9): protocol: read tcp 188.166.225.170:56458->145.79.227.22:4510: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 9 08:42:14.266666573 (833h13m14.137749618s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266732248 (833h13m14.137815303s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266745052 (833h13m14.137828127s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266905067 (833h13m14.137988102s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266913643 (833h13m14.137996699s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266932269 (833h13m14.138015314s) [seg_pos={0 0}, upload=728822]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 08:42:14.267007192 (833h13m14.138090227s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 08:42:14.267013323 (833h13m14.138096368s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 08:42:14.561620288 (833h13m14.432703333s) [upload=728822]: Done uploading segments... Jan 9 08:42:14.561646929 (833h13m14.432729974s) [upload=728822]: Upload finished. err: <nil> Duration: 5.911092724s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 47 out of 60 is failed (threshold=6) [X.XXXX..X.XX.XXXXXXXX.XXXXXXXXX.XXXXXXXXXX.XXXX.XXXXX.XX...X] duration: 5.000822728s trace: https://jaeger.dev.storj.io/trace/2205903894050ab9 FAIL (2026-01-09 09:17:36 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000906617s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [............X.X..................X..........................] duration: 18.86767718s trace: https://jaeger.dev.storj.io/trace/32dce7166c4df329 FAIL (2026-01-09 09:17:15 UTC) orbiter warning: context deadline exceeded Duration: 18.867767231s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [........X.......X.X...XX.XX...X...........XXXX..............] duration: 19.065441202s trace: https://jaeger.dev.storj.io/trace/41529e7a38992b5b FAIL (2026-01-09 09:17:12 UTC) orbiter warning: context deadline exceeded Duration: 19.06546698s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 47 out of 60 is failed (threshold=6) [XX.XXXX..X.XX.XXXXXXXX.XXXXXXXXX.XXXXXXXXXX.XXXX.XXXXX.XX...] duration: 3.915317679s trace: https://jaeger.dev.storj.io/trace/6e29b9e863704824 FAIL (2026-01-09 09:17:08 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.915345011s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.86126374s trace: https://jaeger.dev.storj.io/trace/9c10d1cea23ecd9 FAIL (2026-01-09 09:17:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12HWX3mC14qJg7WdCetaBtS2GcGQFhzSaqUbKrYFBHZKdyFAAgX, Piece ID: ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA) context deadline exceeded; piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A) context deadline exceeded; piecestore: (Node ID: 1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, Piece ID: 3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ) context deadline exceeded ======LOGS====== Jan 9 08:42:09.786209491 (833h13m9.657292536s) [upload=728822]: Starting upload Jan 9 08:42:09.786265998 (833h13m9.657349033s) [upload=728822]: Done starting upload Jan 9 08:42:09.786338025 (833h13m9.65742106s) [upload=728822]: Uploading segments... Jan 9 08:42:09.786470407 (833h13m9.657553443s) [upload=728822]: Got next segment. Inline: false Jan 9 08:42:09.786499243 (833h13m9.657582278s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 08:42:09.786503601 (833h13m9.657586636s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 08:42:10.028227298 (833h13m9.899310333s) [seg_pos={0 0}, upload=728822]: Begin upload segment... Jan 9 08:42:10.028276782 (833h13m9.899359817s) [seg_pos={0 0}, upload=728822]: Done begin upload segment. Jan 9 08:42:10.028298404 (833h13m9.899381439s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028312550 (833h13m9.899395585s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028351725 (833h13m9.89943477s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028742609 (833h13m9.899825664s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029058711 (833h13m9.900141756s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029428165 (833h13m9.9005112s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029762632 (833h13m9.900845667s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030105314 (833h13m9.901188379s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030547576 (833h13m9.901630621s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030977594 (833h13m9.90206063s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.031805010 (833h13m9.902888055s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.032269375 (833h13m9.90335242s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.033511369 (833h13m9.904594415s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.034667751 (833h13m9.905750796s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.035948640 (833h13m9.907031685s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.037387751 (833h13m9.908470786s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:12.640854837 (833h13m12.511937872s) [upload=728822]: Next returned nil segment Jan 9 08:42:12.640867422 (833h13m12.511950457s) [upload=728822]: Waiting for error group managing segments... Jan 9 08:42:13.046761163 (833h13m12.917844209s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.084597619 (833h13m12.955680664s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.092874369 (833h13m12.963957404s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.150407240 (833h13m13.021490275s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.159991590 (833h13m13.031074625s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.176804004 (833h13m13.047887039s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.182523755 (833h13m13.0536068s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.979222818 (833h13m13.850305863s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.006239722 (833h13m13.877322757s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.059101749 (833h13m13.930184795s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.140860029 (833h13m14.011943074s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.186445952 (833h13m14.057528987s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266335753 (833h13m14.137418788s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266359047 (833h13m14.137442092s) [seg_pos={0 0}, upload=728822]: Segment reached optimal threshold of 13 pieces. Jan 9 08:42:14.266656384 (833h13m14.137739429s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9): protocol: read tcp 188.166.225.170:56458->145.79.227.22:4510: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 9 08:42:14.266666573 (833h13m14.137749618s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266732248 (833h13m14.137815303s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266745052 (833h13m14.137828127s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266905067 (833h13m14.137988102s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266913643 (833h13m14.137996699s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266932269 (833h13m14.138015314s) [seg_pos={0 0}, upload=728822]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 08:42:14.267007192 (833h13m14.138090227s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 08:42:14.267013323 (833h13m14.138096368s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 08:42:14.561620288 (833h13m14.432703333s) [upload=728822]: Done uploading segments... Jan 9 08:42:14.561646929 (833h13m14.432729974s) [upload=728822]: Upload finished. err: <nil> Duration: 5.861288617s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.............X.X..................X.........................] duration: 22.708225907s trace: https://jaeger.dev.storj.io/trace/32dce7166c4df329 FAIL (2026-01-09 09:16:40 UTC) orbiter warning: context deadline exceeded Duration: 22.708247417s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 48 out of 60 is failed (threshold=6) [XXX.XXXX..X.XX.XXXXXXXX.XXXXXXXXX.XXXXXXXXXX.XXXX.XXXXX.XX..] duration: 3.710303587s trace: https://jaeger.dev.storj.io/trace/6e29b9e863704824 FAIL (2026-01-09 09:16:08 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.710354024s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.........X.......X.X...XX.XX...X...........XXXX.............] duration: 22.026072995s trace: https://jaeger.dev.storj.io/trace/41529e7a38992b5b FAIL (2026-01-09 09:16:05 UTC) orbiter warning: context deadline exceeded Duration: 22.026113773s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.800424257s trace: https://jaeger.dev.storj.io/trace/9c10d1cea23ecd9 FAIL (2026-01-09 09:16:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12HWX3mC14qJg7WdCetaBtS2GcGQFhzSaqUbKrYFBHZKdyFAAgX, Piece ID: ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA) context deadline exceeded; piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A) context deadline exceeded; piecestore: (Node ID: 1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, Piece ID: 3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ) context deadline exceeded ======LOGS====== Jan 9 08:42:09.786209491 (833h13m9.657292536s) [upload=728822]: Starting upload Jan 9 08:42:09.786265998 (833h13m9.657349033s) [upload=728822]: Done starting upload Jan 9 08:42:09.786338025 (833h13m9.65742106s) [upload=728822]: Uploading segments... Jan 9 08:42:09.786470407 (833h13m9.657553443s) [upload=728822]: Got next segment. Inline: false Jan 9 08:42:09.786499243 (833h13m9.657582278s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 08:42:09.786503601 (833h13m9.657586636s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 08:42:10.028227298 (833h13m9.899310333s) [seg_pos={0 0}, upload=728822]: Begin upload segment... Jan 9 08:42:10.028276782 (833h13m9.899359817s) [seg_pos={0 0}, upload=728822]: Done begin upload segment. Jan 9 08:42:10.028298404 (833h13m9.899381439s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028312550 (833h13m9.899395585s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028351725 (833h13m9.89943477s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028742609 (833h13m9.899825664s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029058711 (833h13m9.900141756s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029428165 (833h13m9.9005112s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029762632 (833h13m9.900845667s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030105314 (833h13m9.901188379s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030547576 (833h13m9.901630621s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030977594 (833h13m9.90206063s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.031805010 (833h13m9.902888055s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.032269375 (833h13m9.90335242s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.033511369 (833h13m9.904594415s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.034667751 (833h13m9.905750796s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.035948640 (833h13m9.907031685s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.037387751 (833h13m9.908470786s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:12.640854837 (833h13m12.511937872s) [upload=728822]: Next returned nil segment Jan 9 08:42:12.640867422 (833h13m12.511950457s) [upload=728822]: Waiting for error group managing segments... Jan 9 08:42:13.046761163 (833h13m12.917844209s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.084597619 (833h13m12.955680664s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.092874369 (833h13m12.963957404s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.150407240 (833h13m13.021490275s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.159991590 (833h13m13.031074625s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.176804004 (833h13m13.047887039s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.182523755 (833h13m13.0536068s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.979222818 (833h13m13.850305863s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.006239722 (833h13m13.877322757s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.059101749 (833h13m13.930184795s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.140860029 (833h13m14.011943074s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.186445952 (833h13m14.057528987s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266335753 (833h13m14.137418788s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266359047 (833h13m14.137442092s) [seg_pos={0 0}, upload=728822]: Segment reached optimal threshold of 13 pieces. Jan 9 08:42:14.266656384 (833h13m14.137739429s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9): protocol: read tcp 188.166.225.170:56458->145.79.227.22:4510: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 9 08:42:14.266666573 (833h13m14.137749618s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266732248 (833h13m14.137815303s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266745052 (833h13m14.137828127s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266905067 (833h13m14.137988102s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266913643 (833h13m14.137996699s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266932269 (833h13m14.138015314s) [seg_pos={0 0}, upload=728822]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 08:42:14.267007192 (833h13m14.138090227s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 08:42:14.267013323 (833h13m14.138096368s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 08:42:14.561620288 (833h13m14.432703333s) [upload=728822]: Done uploading segments... Jan 9 08:42:14.561646929 (833h13m14.432729974s) [upload=728822]: Upload finished. err: <nil> Duration: 5.800454815s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 49 out of 60 is failed (threshold=6) [XXXX.XXXX..X.XX.XXXXXXXX.XXXXXXXXX.XXXXXXXXXX.XXXX.XXXXX.XX.] duration: 4.294843991s trace: https://jaeger.dev.storj.io/trace/6e29b9e863704824 FAIL (2026-01-09 09:15:40 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.294872384s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.886639932s trace: https://jaeger.dev.storj.io/trace/9c10d1cea23ecd9 FAIL (2026-01-09 09:15:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12HWX3mC14qJg7WdCetaBtS2GcGQFhzSaqUbKrYFBHZKdyFAAgX, Piece ID: ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA) context deadline exceeded; piecestore: (Node ID: 12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ, Piece ID: OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A) context deadline exceeded; piecestore: (Node ID: 1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, Piece ID: 3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ) context deadline exceeded ======LOGS====== Jan 9 08:42:09.786209491 (833h13m9.657292536s) [upload=728822]: Starting upload Jan 9 08:42:09.786265998 (833h13m9.657349033s) [upload=728822]: Done starting upload Jan 9 08:42:09.786338025 (833h13m9.65742106s) [upload=728822]: Uploading segments... Jan 9 08:42:09.786470407 (833h13m9.657553443s) [upload=728822]: Got next segment. Inline: false Jan 9 08:42:09.786499243 (833h13m9.657582278s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 9 08:42:09.786503601 (833h13m9.657586636s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 9 08:42:10.028227298 (833h13m9.899310333s) [seg_pos={0 0}, upload=728822]: Begin upload segment... Jan 9 08:42:10.028276782 (833h13m9.899359817s) [seg_pos={0 0}, upload=728822]: Done begin upload segment. Jan 9 08:42:10.028298404 (833h13m9.899381439s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028312550 (833h13m9.899395585s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028351725 (833h13m9.89943477s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.028742609 (833h13m9.899825664s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029058711 (833h13m9.900141756s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029428165 (833h13m9.9005112s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.029762632 (833h13m9.900845667s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030105314 (833h13m9.901188379s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030547576 (833h13m9.901630621s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.030977594 (833h13m9.90206063s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.031805010 (833h13m9.902888055s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.032269375 (833h13m9.90335242s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.033511369 (833h13m9.904594415s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.034667751 (833h13m9.905750796s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.035948640 (833h13m9.907031685s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:10.037387751 (833h13m9.908470786s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Uploading piece... Jan 9 08:42:12.640854837 (833h13m12.511937872s) [upload=728822]: Next returned nil segment Jan 9 08:42:12.640867422 (833h13m12.511950457s) [upload=728822]: Waiting for error group managing segments... Jan 9 08:42:13.046761163 (833h13m12.917844209s) [noise=true , address=77.90.31.14:4661 , piece_id=PYCHZK35FDULTPRUX56O2XALZN2XTFGORUTHN2ZIVWX27BJKWXYQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.084597619 (833h13m12.955680664s) [noise=true , address=145.79.227.7:4293 , piece_id=3Q3CS35TVAMQXQ3Q33T2LCTXRZDKZHEVIH7GHK25NUL3IRYN57OA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.092874369 (833h13m12.963957404s) [noise=true , address=145.79.227.40:4050 , piece_id=ILAJL7ASFS3OYIM2227Q2HA2Z23C6LW5DKZZXXVHKUALQQKPI67A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.150407240 (833h13m13.021490275s) [noise=true , address=145.79.227.43:4157 , piece_id=YD3RUJRZILONLPJJ3NV7XWZPVTTKA2PHPURHMCGP42CDD7M6Q3XQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.159991590 (833h13m13.031074625s) [noise=true , address=145.79.227.3:4149 , piece_id=BDUXIWLM6JNKEFPXNFC6KMUNYQDVIQO662XY5OZ25AG74NM2YVGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.176804004 (833h13m13.047887039s) [noise=true , address=145.79.227.50:4373 , piece_id=VT6G6LWASBYCQUTD26KA5SKW527KU4T4CKTEBCJHIUMVEP6QZNIA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.182523755 (833h13m13.0536068s) [noise=true , address=145.79.227.48:4303 , piece_id=6OEADTNLO4BZQ2NS24VIXSPNHTDILXYJI67CVI7EMJTLWIIAENAA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:13.979222818 (833h13m13.850305863s) [noise=true , address=145.79.227.49:4338 , piece_id=IEVY6XYIAZAGPNB6LX232YBMJ4Q6NHKNCSDKDUTCQZPWTRPW3D6Q, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.006239722 (833h13m13.877322757s) [noise=true , address=145.79.227.51:4410 , piece_id=ATCJW3G3MVAW6HXOZCMBOWNKRF2XVZBUA5QNB6APPCHNHL5SJ4NA, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.059101749 (833h13m13.930184795s) [noise=true , address=145.79.227.47:4265 , piece_id=OPTPM6AROHT5GTPNPR5LQEZSBKBWROFJQOTUQOPJKAATM35Z2J2A, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.140860029 (833h13m14.011943074s) [noise=true , address=45.91.116.13:4154 , piece_id=3CHWY2M2OGXOZ4O67NGUJ7K4EF4LKA5PJV2GS2OANJ2IAGMKTIOQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.186445952 (833h13m14.057528987s) [noise=true , address=45.91.116.10:4082 , piece_id=XWRCH22HBUTCZK5TYMK3CRZZPG57JEHZITCPW4Z7EADXX25C6PMQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266335753 (833h13m14.137418788s) [noise=true , address=145.79.227.19:4078 , piece_id=WXAFKODDG46BT6JEH5G56PEK44BLFNJINDFSODSBHY6YNQ4GGUTQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: <nil> Jan 9 08:42:14.266359047 (833h13m14.137442092s) [seg_pos={0 0}, upload=728822]: Segment reached optimal threshold of 13 pieces. Jan 9 08:42:14.266656384 (833h13m14.137739429s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9): protocol: read tcp 188.166.225.170:56458->145.79.227.22:4510: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 9 08:42:14.266666573 (833h13m14.137749618s) [noise=true , address=145.79.227.22:4510 , piece_id=ATZ72JVSAWPXUPTMXHAO7TNLXNUHKHIUFPOZ5R42P4LRALW2CFGQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266732248 (833h13m14.137815303s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129gPpguzPP8UDJEMDM9YPuKXiDVLhijvpeb6VYDMdzyccof35H): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266745052 (833h13m14.137828127s) [noise=true , address=45.91.116.8:4046 , piece_id=JPMTUQIAH3HER7LGI37HYXNK2SISRA5BNE73BH65GMPMAEQZJNLQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266905067 (833h13m14.137988102s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jan 9 08:42:14.266913643 (833h13m14.137996699s) [noise=true , address=145.79.227.11:4731 , piece_id=P7NS5NNGXN7JWUMCACZXL5NW7QTODRYIA377QQMNXNZDPDW43PCQ, seg_pos={0 0}, upload=728822]: Long tail cancellation due to optimal threshold Jan 9 08:42:14.266932269 (833h13m14.138015314s) [seg_pos={0 0}, upload=728822]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jan 9 08:42:14.267007192 (833h13m14.138090227s) [upload=728822]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jan 9 08:42:14.267013323 (833h13m14.138096368s) [upload=728822]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jan 9 08:42:14.561620288 (833h13m14.432703333s) [upload=728822]: Done uploading segments... Jan 9 08:42:14.561646929 (833h13m14.432729974s) [upload=728822]: Upload finished. err: <nil> Duration: 5.886664027s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)