Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 14 out of 60 is failed (threshold=3) [XX...XX..XX..........................X.XXX.X.X......X....X..] duration: 8.250632748s trace: https://jaeger.dev.storj.io/trace/7439900a15bb5b43 FAIL (2026-05-15 22:42:34 UTC) orbiter warning: context deadline exceeded Duration: 8.250644872s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 9.027173259s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:42:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 9.027195701s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [...X.............X.......X.....X.....X....XX................] duration: 3.848826817s trace: https://jaeger.dev.storj.io/trace/1cb4c90b9422be55 FAIL (2026-05-15 22:41:50 UTC) orbiter warning: context deadline exceeded Duration: 3.84889135s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 14 out of 60 is failed (threshold=3) [.XX...XX..XX..........................X.XXX.X.X......X....X.] duration: 16.428333869s trace: https://jaeger.dev.storj.io/trace/7439900a15bb5b43 FAIL (2026-05-15 22:41:29 UTC) orbiter warning: context deadline exceeded Duration: 16.428346503s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 14 out of 60 is failed (threshold=3) [..XX...XX..XX..........................X.XXX.X.X......X....X] duration: 1m0.001342452s trace: https://jaeger.dev.storj.io/trace/7439900a15bb5b43 FAIL (2026-05-15 22:41:13 UTC) orbiter warning: context deadline exceeded Duration: 1m0.001363593s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.466859942s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:41:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 6.466880092s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [....X.............X.......X.....X.....X....XX...............] duration: 4.075163336s trace: https://jaeger.dev.storj.io/trace/1cb4c90b9422be55 FAIL (2026-05-15 22:40:43 UTC) orbiter warning: context deadline exceeded Duration: 4.075187411s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [.....X.............X.......X.....X.....X....XX..............] duration: 3.897689449s trace: https://jaeger.dev.storj.io/trace/1cb4c90b9422be55 FAIL (2026-05-15 22:40:27 UTC) orbiter warning: context deadline exceeded Duration: 3.897707343s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 6.273081503s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:40:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 6.273106942s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 13 out of 60 is failed (threshold=3) [...XX...XX..XX..........................X.XXX.X.X......X....] duration: 25.916225304s trace: https://jaeger.dev.storj.io/trace/58193e8a335c336a FAIL (2026-05-15 22:39:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 25.916344442s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [......X.............X.......X.....X.....X....XX.............] duration: 4.19402671s trace: https://jaeger.dev.storj.io/trace/1cb4c90b9422be55 FAIL (2026-05-15 22:39:27 UTC) orbiter warning: context deadline exceeded Duration: 4.194092345s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 6.557217131s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:39:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 6.557241898s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [.......X.............X.......X.....X.....X....XX............] duration: 3.833373959s trace: https://jaeger.dev.storj.io/trace/1cb4c90b9422be55 FAIL (2026-05-15 22:38:57 UTC) orbiter warning: context deadline exceeded Duration: 3.833397083s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 8.737321204s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:38:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 8.737349236s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 6.74002724s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:38:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 6.740046446s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 13 out of 60 is failed (threshold=3) [....XX...XX..XX..........................X.XXX.X.X......X...] duration: 32.020362977s trace: https://jaeger.dev.storj.io/trace/58193e8a335c336a FAIL (2026-05-15 22:38:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 32.020394637s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [X.......X.............X.......X.....X.....X....XX...........] duration: 4.302951469s trace: https://jaeger.dev.storj.io/trace/1cb4c90b9422be55 FAIL (2026-05-15 22:37:47 UTC) orbiter warning: context deadline exceeded Duration: 4.302975475s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 7.843432284s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:37:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 7.843447111s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 10.724871523s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:37:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 10.724921528s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 13 out of 60 is failed (threshold=3) [.....XX...XX..XX..........................X.XXX.X.X......X..] duration: 7.432996979s trace: https://jaeger.dev.storj.io/trace/58193e8a335c336a FAIL (2026-05-15 22:36:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.433019583s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [.X.......X.............X.......X.....X.....X....XX..........] duration: 4.42373904s trace: https://jaeger.dev.storj.io/trace/1cb4c90b9422be55 FAIL (2026-05-15 22:36:51 UTC) orbiter warning: context deadline exceeded Duration: 4.42378092s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 13 out of 60 is failed (threshold=3) [......XX...XX..XX..........................X.XXX.X.X......X.] duration: 7.566439317s trace: https://jaeger.dev.storj.io/trace/58193e8a335c336a FAIL (2026-05-15 22:36:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.566453503s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 13 out of 60 is failed (threshold=3) [.......XX...XX..XX..........................X.XXX.X.X......X] duration: 1m0.001367306s trace: https://jaeger.dev.storj.io/trace/58193e8a335c336a FAIL (2026-05-15 22:36:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001392836s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [..X.......X.............X.......X.....X.....X....XX.........] duration: 4.185139967s trace: https://jaeger.dev.storj.io/trace/1cb4c90b9422be55 FAIL (2026-05-15 22:36:03 UTC) orbiter warning: context deadline exceeded Duration: 4.185165897s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.894449942s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:35:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 5.894477465s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 7.518937094s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:34:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 7.518954566s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [...X.......X.............X.......X.....X.....X....XX........] duration: 5.000689852s trace: https://jaeger.dev.storj.io/trace/1cb4c90b9422be55 FAIL (2026-05-15 22:34:57 UTC) orbiter warning: context deadline exceeded Duration: 5.000708929s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 12 out of 60 is failed (threshold=3) [........XX...XX..XX..........................X.XXX.X.X......] duration: 40.254868322s trace: https://jaeger.dev.storj.io/trace/74490d62781b0ea8 FAIL (2026-05-15 22:34:40 UTC) orbiter warning: context deadline exceeded Duration: 40.254880475s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 12 out of 60 is failed (threshold=3) [.........XX...XX..XX..........................X.XXX.X.X.....] duration: 45.308999484s trace: https://jaeger.dev.storj.io/trace/74490d62781b0ea8 FAIL (2026-05-15 22:34:00 UTC) orbiter warning: context deadline exceeded Duration: 45.309041914s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 9.72822178s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:33:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 9.728246006s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 9 out of 60 is failed (threshold=6) [X...X.......X.............X.......X.....X.....X....XX.......] duration: 4.094240362s trace: https://jaeger.dev.storj.io/trace/1cb4c90b9422be55 FAIL (2026-05-15 22:33:51 UTC) orbiter warning: context deadline exceeded Duration: 4.094263307s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 11.506620612s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:33:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 11.506647644s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 9 out of 60 is failed (threshold=6) [.X...X.......X.............X.......X.....X.....X....XX......] duration: 5.001507822s trace: https://jaeger.dev.storj.io/trace/1cb4c90b9422be55 FAIL (2026-05-15 22:33:31 UTC) orbiter warning: context deadline exceeded Duration: 5.001524484s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 7.374350683s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:32:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 7.374378506s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 12 out of 60 is failed (threshold=3) [..........XX...XX..XX..........................X.XXX.X.X....] duration: 36.987907156s trace: https://jaeger.dev.storj.io/trace/74490d62781b0ea8 FAIL (2026-05-15 22:32:41 UTC) orbiter warning: context deadline exceeded Duration: 36.987925511s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 8.291344812s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:32:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 8.29136913s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 9 out of 60 is failed (threshold=6) [..X...X.......X.............X.......X.....X.....X....XX.....] duration: 3.940382892s trace: https://jaeger.dev.storj.io/trace/1cb4c90b9422be55 FAIL (2026-05-15 22:32:16 UTC) orbiter warning: context deadline exceeded Duration: 3.940405305s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 12 out of 60 is failed (threshold=3) [...........XX...XX..XX..........................X.XXX.X.X...] duration: 33.732791207s trace: https://jaeger.dev.storj.io/trace/74490d62781b0ea8 FAIL (2026-05-15 22:32:04 UTC) orbiter warning: context deadline exceeded Duration: 33.732805623s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 9.174093371s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:31:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 9.174116094s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 9 out of 60 is failed (threshold=6) [...X...X.......X.............X.......X.....X.....X....XX....] duration: 3.942788692s trace: https://jaeger.dev.storj.io/trace/1cb4c90b9422be55 FAIL (2026-05-15 22:31:32 UTC) orbiter warning: context deadline exceeded Duration: 3.942818319s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 12 out of 60 is failed (threshold=3) [............XX...XX..XX..........................X.XXX.X.X..] duration: 24.161373783s trace: https://jaeger.dev.storj.io/trace/74490d62781b0ea8 FAIL (2026-05-15 22:31:30 UTC) orbiter warning: context deadline exceeded Duration: 24.161385074s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 12 out of 60 is failed (threshold=3) [.............XX...XX..XX..........................X.XXX.X.X.] duration: 55.400084739s trace: https://jaeger.dev.storj.io/trace/74490d62781b0ea8 FAIL (2026-05-15 22:31:05 UTC) orbiter warning: context deadline exceeded Duration: 55.40010635s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 6.679912827s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:30:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 6.679937783s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 10 out of 60 is failed (threshold=6) [X...X...X.......X.............X.......X.....X.....X....XX...] duration: 4.060314625s trace: https://jaeger.dev.storj.io/trace/1cb4c90b9422be55 FAIL (2026-05-15 22:30:33 UTC) orbiter warning: context deadline exceeded Duration: 4.060339323s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 10.979890164s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:30:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 10.979921123s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 12.578231118s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:30:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 12.578255054s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 12 out of 60 is failed (threshold=3) [..............XX...XX..XX..........................X.XXX.X.X] duration: 1m0.000534298s trace: https://jaeger.dev.storj.io/trace/74490d62781b0ea8 FAIL (2026-05-15 22:30:09 UTC) orbiter warning: context deadline exceeded Duration: 1m0.000554576s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 10 out of 60 is failed (threshold=6) [.X...X...X.......X.............X.......X.....X.....X....XX..] duration: 3.896522067s trace: https://jaeger.dev.storj.io/trace/1cb4c90b9422be55 FAIL (2026-05-15 22:29:48 UTC) orbiter warning: context deadline exceeded Duration: 3.896543287s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 8.457265612s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:29:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 8.457297652s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 12.962908873s trace: https://jaeger.dev.storj.io/trace/15d3b92a22e517da FAIL (2026-05-15 22:28:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments... May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment... May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment. May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece... May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments... May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil> May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments... May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil> Duration: 12.962934381s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)