Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [X.......X.............X.......X.....X.....X....XX...........] duration: 4.302951469s trace: https://jaeger.dev.storj.io/trace/1cb4c90b9422be55 FAIL (2026-05-15 22:37:47 UTC) orbiter warning: context deadline exceeded Duration: 4.302975475s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 7.843432284s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:37:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 7.843447111s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 10.724871523s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:37:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 10.724921528s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 13 out of 60 is failed (threshold=3) [.....XX...XX..XX..........................X.XXX.X.X......X..] duration: 7.432996979s trace: https://jaeger.dev.storj.io/trace/58193e8a335c336a FAIL (2026-05-15 22:36:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.433019583s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [.X.......X.............X.......X.....X.....X....XX..........] duration: 4.42373904s trace: https://jaeger.dev.storj.io/trace/1cb4c90b9422be55 FAIL (2026-05-15 22:36:51 UTC) orbiter warning: context deadline exceeded Duration: 4.42378092s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 13 out of 60 is failed (threshold=3) [......XX...XX..XX..........................X.XXX.X.X......X.] duration: 7.566439317s trace: https://jaeger.dev.storj.io/trace/58193e8a335c336a FAIL (2026-05-15 22:36:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.566453503s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 13 out of 60 is failed (threshold=3) [.......XX...XX..XX..........................X.XXX.X.X......X] duration: 1m0.001367306s trace: https://jaeger.dev.storj.io/trace/58193e8a335c336a FAIL (2026-05-15 22:36:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001392836s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [..X.......X.............X.......X.....X.....X....XX.........] duration: 4.185139967s trace: https://jaeger.dev.storj.io/trace/1cb4c90b9422be55 FAIL (2026-05-15 22:36:03 UTC) orbiter warning: context deadline exceeded Duration: 4.185165897s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.894449942s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:35:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 5.894477465s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 7.518937094s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:34:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 7.518954566s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [...X.......X.............X.......X.....X.....X....XX........] duration: 5.000689852s trace: https://jaeger.dev.storj.io/trace/1cb4c90b9422be55 FAIL (2026-05-15 22:34:57 UTC) orbiter warning: context deadline exceeded Duration: 5.000708929s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 12 out of 60 is failed (threshold=3) [........XX...XX..XX..........................X.XXX.X.X......] duration: 40.254868322s trace: https://jaeger.dev.storj.io/trace/74490d62781b0ea8 FAIL (2026-05-15 22:34:40 UTC) orbiter warning: context deadline exceeded Duration: 40.254880475s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 12 out of 60 is failed (threshold=3) [.........XX...XX..XX..........................X.XXX.X.X.....] duration: 45.308999484s trace: https://jaeger.dev.storj.io/trace/74490d62781b0ea8 FAIL (2026-05-15 22:34:00 UTC) orbiter warning: context deadline exceeded Duration: 45.309041914s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 9.72822178s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:33:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 9.728246006s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 9 out of 60 is failed (threshold=6) [X...X.......X.............X.......X.....X.....X....XX.......] duration: 4.094240362s trace: https://jaeger.dev.storj.io/trace/1cb4c90b9422be55 FAIL (2026-05-15 22:33:51 UTC) orbiter warning: context deadline exceeded Duration: 4.094263307s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 11.506620612s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:33:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 11.506647644s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 9 out of 60 is failed (threshold=6) [.X...X.......X.............X.......X.....X.....X....XX......] duration: 5.001507822s trace: https://jaeger.dev.storj.io/trace/1cb4c90b9422be55 FAIL (2026-05-15 22:33:31 UTC) orbiter warning: context deadline exceeded Duration: 5.001524484s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 7.374350683s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:32:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 7.374378506s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 12 out of 60 is failed (threshold=3) [..........XX...XX..XX..........................X.XXX.X.X....] duration: 36.987907156s trace: https://jaeger.dev.storj.io/trace/74490d62781b0ea8 FAIL (2026-05-15 22:32:41 UTC) orbiter warning: context deadline exceeded Duration: 36.987925511s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 8.291344812s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:32:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 8.29136913s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 9 out of 60 is failed (threshold=6) [..X...X.......X.............X.......X.....X.....X....XX.....] duration: 3.940382892s trace: https://jaeger.dev.storj.io/trace/1cb4c90b9422be55 FAIL (2026-05-15 22:32:16 UTC) orbiter warning: context deadline exceeded Duration: 3.940405305s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 12 out of 60 is failed (threshold=3) [...........XX...XX..XX..........................X.XXX.X.X...] duration: 33.732791207s trace: https://jaeger.dev.storj.io/trace/74490d62781b0ea8 FAIL (2026-05-15 22:32:04 UTC) orbiter warning: context deadline exceeded Duration: 33.732805623s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 9.174093371s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:31:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 9.174116094s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 9 out of 60 is failed (threshold=6) [...X...X.......X.............X.......X.....X.....X....XX....] duration: 3.942788692s trace: https://jaeger.dev.storj.io/trace/1cb4c90b9422be55 FAIL (2026-05-15 22:31:32 UTC) orbiter warning: context deadline exceeded Duration: 3.942818319s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 12 out of 60 is failed (threshold=3) [............XX...XX..XX..........................X.XXX.X.X..] duration: 24.161373783s trace: https://jaeger.dev.storj.io/trace/74490d62781b0ea8 FAIL (2026-05-15 22:31:30 UTC) orbiter warning: context deadline exceeded Duration: 24.161385074s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 12 out of 60 is failed (threshold=3) [.............XX...XX..XX..........................X.XXX.X.X.] duration: 55.400084739s trace: https://jaeger.dev.storj.io/trace/74490d62781b0ea8 FAIL (2026-05-15 22:31:05 UTC) orbiter warning: context deadline exceeded Duration: 55.40010635s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 6.679912827s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:30:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 6.679937783s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 10 out of 60 is failed (threshold=6) [X...X...X.......X.............X.......X.....X.....X....XX...] duration: 4.060314625s trace: https://jaeger.dev.storj.io/trace/1cb4c90b9422be55 FAIL (2026-05-15 22:30:33 UTC) orbiter warning: context deadline exceeded Duration: 4.060339323s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 10.979890164s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:30:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 10.979921123s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 12.578231118s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:30:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 12.578255054s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 12 out of 60 is failed (threshold=3) [..............XX...XX..XX..........................X.XXX.X.X] duration: 1m0.000534298s trace: https://jaeger.dev.storj.io/trace/74490d62781b0ea8 FAIL (2026-05-15 22:30:09 UTC) orbiter warning: context deadline exceeded Duration: 1m0.000554576s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 10 out of 60 is failed (threshold=6) [.X...X...X.......X.............X.......X.....X.....X....XX..] duration: 3.896522067s trace: https://jaeger.dev.storj.io/trace/1cb4c90b9422be55 FAIL (2026-05-15 22:29:48 UTC) orbiter warning: context deadline exceeded Duration: 3.896543287s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 8.457265612s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:29:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 8.457297652s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 12.962908873s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:28:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 12.962934381s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 11 out of 60 is failed (threshold=3) [...............XX...XX..XX..........................X.XXX.X.] duration: 18.410698542s trace: https://jaeger.dev.storj.io/trace/20fe68f984fc2d4c FAIL (2026-05-15 22:28:51 UTC) orbiter warning: context deadline exceeded Duration: 18.410710985s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 11 out of 60 is failed (threshold=6) [X.X...X...X.......X.............X.......X.....X.....X....XX.] duration: 4.763261117s trace: https://jaeger.dev.storj.io/trace/1cb4c90b9422be55 FAIL (2026-05-15 22:28:46 UTC) orbiter warning: context deadline exceeded Duration: 4.763281497s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.002141146s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:28:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 30.002163077s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 11 out of 60 is failed (threshold=3) [................XX...XX..XX..........................X.XXX.X] duration: 1m0.001191733s trace: https://jaeger.dev.storj.io/trace/20fe68f984fc2d4c FAIL (2026-05-15 22:28:32 UTC) orbiter warning: context deadline exceeded Duration: 1m0.001204236s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 11 out of 60 is failed (threshold=6) [.X.X...X...X.......X.............X.......X.....X.....X....XX] duration: 5.001375818s trace: https://jaeger.dev.storj.io/trace/1cb4c90b9422be55 FAIL (2026-05-15 22:28:13 UTC) orbiter warning: context deadline exceeded Duration: 5.001393723s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 10 out of 60 is failed (threshold=6) [..X.X...X...X.......X.............X.......X.....X.....X....X] duration: 5.001202558s trace: https://jaeger.dev.storj.io/trace/4e82b0628ad1238e FAIL (2026-05-15 22:27:40 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001223027s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 10 out of 60 is failed (threshold=3) [.................XX...XX..XX..........................X.XXX.] duration: 42.089710321s trace: https://jaeger.dev.storj.io/trace/6cac86a8a124846c FAIL (2026-05-15 22:27:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 42.089735489s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 9 out of 60 is failed (threshold=6) [...X.X...X...X.......X.............X.......X.....X.....X....] duration: 4.265174469s trace: https://jaeger.dev.storj.io/trace/7954c8e12256e9c5 FAIL (2026-05-15 22:27:05 UTC) orbiter warning: context deadline exceeded Duration: 4.265196992s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 10 out of 60 is failed (threshold=3) [..................XX...XX..XX..........................X.XXX] duration: 1m0.001236819s trace: https://jaeger.dev.storj.io/trace/6cac86a8a124846c FAIL (2026-05-15 22:26:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001248131s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 9 out of 60 is failed (threshold=6) [....X.X...X...X.......X.............X.......X.....X.....X...] duration: 3.905978461s trace: https://jaeger.dev.storj.io/trace/7954c8e12256e9c5 FAIL (2026-05-15 22:26:02 UTC) orbiter warning: context deadline exceeded Duration: 3.906001785s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 9 out of 60 is failed (threshold=3) [...................XX...XX..XX..........................X.XX] duration: 1m0.000794999s trace: https://jaeger.dev.storj.io/trace/228f705c37527110 FAIL (2026-05-15 22:25:43 UTC) orbiter warning: context deadline exceeded Duration: 1m0.000805008s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 9 out of 60 is failed (threshold=6) [.....X.X...X...X.......X.............X.......X.....X.....X..] duration: 4.308442335s trace: https://jaeger.dev.storj.io/trace/7954c8e12256e9c5 FAIL (2026-05-15 22:25:21 UTC) orbiter warning: context deadline exceeded Duration: 4.308469196s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 9 out of 60 is failed (threshold=6) [......X.X...X...X.......X.............X.......X.....X.....X.] duration: 3.803385736s trace: https://jaeger.dev.storj.io/trace/7954c8e12256e9c5 FAIL (2026-05-15 22:24:53 UTC) orbiter warning: context deadline exceeded Duration: 3.803406937s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 8 out of 60 is failed (threshold=3) [....................XX...XX..XX..........................X.X] duration: 1m0.001267839s trace: https://jaeger.dev.storj.io/trace/6a4d76d0e12cfd67 FAIL (2026-05-15 22:24:43 UTC) orbiter warning: context deadline exceeded Duration: 1m0.001289412s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 9 out of 60 is failed (threshold=6) [.......X.X...X...X.......X.............X.......X.....X.....X] duration: 5.001567475s trace: https://jaeger.dev.storj.io/trace/7954c8e12256e9c5 FAIL (2026-05-15 22:24:19 UTC) orbiter warning: context deadline exceeded Duration: 5.001597141s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [........X.X...X...X.......X.............X.......X.....X.....] duration: 4.739576704s trace: https://jaeger.dev.storj.io/trace/8214ed6c56880aa FAIL (2026-05-15 22:23:45 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.739598686s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)