Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.930851018s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:46:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.930876477s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 5.94151099s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:45:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.94154731s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 6.596052036s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:45:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.596143642s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 6.82502879s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:45:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.825048048s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 6.99239768s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:44:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.992425152s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 6.602552471s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:44:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.602576828s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 6.574110507s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:44:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.574136447s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 6.626033088s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:43:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.626075859s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.778830162s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:43:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.778861712s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 6.040924056s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:42:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.040947451s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 6.933508967s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:42:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.933555715s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 6.864810515s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:42:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.864876731s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.817357146s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:41:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.81737017s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 6.627903313s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:41:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.627930114s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.708925287s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:41:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.708953711s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 6.635251919s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:40:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.635275245s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 6.281818489s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:40:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.281910955s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.879059681s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:40:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.879088205s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.74774529s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:39:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.747782421s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.678830117s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:39:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.67884833s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.837416076s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:38:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.837506169s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.883218669s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:38:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.883243356s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 6.340378266s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:38:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.340415627s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.807378862s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:37:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.807398691s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 7.011800154s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:37:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 7.011827716s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.029380785s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:37:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.029408297s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 6.596080342s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:37:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.596110359s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.651944763s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:36:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.651971424s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.579120142s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:36:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.57914053s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.764497622s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:35:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.764521466s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 11.294268136s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:35:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 11.294292924s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.785893719s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:35:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.785911483s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 6.743694698s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:34:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.743720648s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.646597995s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:34:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.646612983s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 7.321395661s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:34:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 7.321424584s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.536983796s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:33:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.537314406s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 10.828499611s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:33:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 10.828524588s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 6.349628053s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:32:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.349673691s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.162687233s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:32:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.162706629s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 6.748567523s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:32:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.748589805s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.897723623s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:31:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.897749792s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.809290827s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:31:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.809378454s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 6.373076631s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:30:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.373106157s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.725984211s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:30:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.725998028s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.720215025s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:30:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.720240224s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.969734055s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:29:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.969764302s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 6.691770991s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:29:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.691807872s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 6.607155256s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:28:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.60730889s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.821037684s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:28:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.821065117s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 6.968190935s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:27:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.968208018s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)