Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.795813914s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:55:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.795851273s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.57097716s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:55:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.571003149s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.446464264s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:55:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.446492097s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.951262959s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:54:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.951283488s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.798092329s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:54:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.798117507s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.471013421s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:53:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.471040174s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 8.54255632s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:52:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.542580568s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.527905058s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:52:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.527932499s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.750702186s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:52:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.750718618s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.566441141s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:51:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.566463313s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.247561601s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:51:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.24758172s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.746864488s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:51:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.74689693s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.151178468s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:50:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.151206371s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 7.11777316s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:50:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.117798748s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.478484666s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:49:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.478508692s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.501245185s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:49:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.501270171s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.333651659s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:49:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.333672528s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 6.984369221s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:49:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.984391502s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.269185774s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:48:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.269212225s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 6.58016556s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:48:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.58019212s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 7.406263687s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:48:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.406277675s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 5.945992069s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:48:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.946028288s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 6.488022336s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:47:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.48804427s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 6.815400556s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:46:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.81542394s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 6.355235839s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:46:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.355258051s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.930851018s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:46:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.930876477s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 5.94151099s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:45:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.94154731s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 6.596052036s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:45:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.596143642s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 6.82502879s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:45:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.825048048s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 6.99239768s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:44:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.992425152s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 6.602552471s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:44:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.602576828s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 6.574110507s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:44:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.574136447s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 6.626033088s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:43:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.626075859s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.778830162s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:43:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.778861712s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 6.040924056s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:42:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.040947451s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 6.933508967s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:42:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.933555715s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 6.864810515s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:42:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.864876731s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.817357146s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:41:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.81737017s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 6.627903313s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:41:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.627930114s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.708925287s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:41:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.708953711s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 6.635251919s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:40:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.635275245s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 6.281818489s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:40:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.281910955s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.879059681s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:40:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.879088205s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.74774529s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:39:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.747782421s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.678830117s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:39:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.67884833s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.837416076s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:38:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.837506169s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.883218669s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:38:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.883243356s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 6.340378266s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:38:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.340415627s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.807378862s trace: https://jaeger.dev.storj.io/trace/40719ae419ee0c7b FAIL (2026-05-15 07:37:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.807398691s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 7.011800154s trace: https://jaeger.dev.storj.io/trace/6cd842384f67522d FAIL (2026-05-15 07:37:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 07:11:17.133292002 (348h23m27.654115747s) [upload=333776]: Starting upload May 15 07:11:17.133342018 (348h23m27.654165763s) [upload=333776]: Done starting upload May 15 07:11:17.133430116 (348h23m27.654253861s) [upload=333776]: Uploading segments... May 15 07:11:17.133508005 (348h23m27.65433174s) [upload=333776]: Got next segment. Inline: false May 15 07:11:17.133538113 (348h23m27.654361858s) [upload=333776]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 07:11:17.133545096 (348h23m27.654368841s) [upload=333776]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 07:11:17.369169107 (348h23m27.889992842s) [seg_pos={0 0}, upload=333776]: Begin upload segment... May 15 07:11:17.369220355 (348h23m27.89004409s) [seg_pos={0 0}, upload=333776]: Done begin upload segment. May 15 07:11:17.369249861 (348h23m27.890073606s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369263687 (348h23m27.890087432s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369296039 (348h23m27.890119784s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.369680944 (348h23m27.890504689s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370096036 (348h23m27.890919781s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370452266 (348h23m27.891276011s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.370749884 (348h23m27.891573619s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371012925 (348h23m27.89183667s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.371277450 (348h23m27.892101195s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.372979648 (348h23m27.893803403s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.373652191 (348h23m27.894475946s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.375751157 (348h23m27.896574902s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.376520906 (348h23m27.897344651s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.378443495 (348h23m27.89926724s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.379461638 (348h23m27.900285383s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:17.384316512 (348h23m27.905140257s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Uploading piece... May 15 07:11:18.638073351 (348h23m29.158897096s) [upload=333776]: Next returned nil segment May 15 07:11:18.638086084 (348h23m29.158909829s) [upload=333776]: Waiting for error group managing segments... May 15 07:11:18.968734326 (348h23m29.489558071s) [noise=true , address=193.149.164.79:4515 , piece_id=KPPRZ55QRWF2ZJ6NYB4UJCXGJF4HVYUXSSLHC4X76OPJM26SJ4FA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:18.983243504 (348h23m29.504067249s) [noise=true , address=193.149.164.78:4467 , piece_id=2BUULIM5RCKUVODZAPLROP53IDXTHMVTSBBYNUWSHSAISSYIXCUA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.244245672 (348h23m29.765069417s) [noise=true , address=145.79.227.10:4420 , piece_id=ONL5U5W2OBJYYHQONYTUHVKARN2TVF7AF2GEQUBRYL3ICAX4TE2A, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.269673465 (348h23m29.79049721s) [noise=true , address=145.79.227.5:4245 , piece_id=VYXJ7MGQKQ4RQVRFPBNTW4WBYO2SSVKEGZ6T4SPEJOYCLK5MSSHA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273168174 (348h23m29.793991919s) [noise=true , address=145.79.227.51:4423 , piece_id=IEFJCGDCMBBXSH7FFEIHB6JTVNMMR3CSZR3O32JSHV5ZV5RVIPPA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.273708305 (348h23m29.79453205s) [noise=true , address=145.79.227.31:4830 , piece_id=UNRZUCNVQYDEMPLMYIO2IDHMDE3WQXS7NQT43W5XVIZCGCKYVPUQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.307772418 (348h23m29.828596163s) [noise=true , address=145.79.227.48:4325 , piece_id=4WAYBH6MPLWBEQ76LFORGMNKGD5IPW3LXDVUE2PEBQ7QH2NJZFQQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.344740434 (348h23m29.865564179s) [noise=true , address=77.90.31.18:4830 , piece_id=SBL25UCJ64HFUC7JV7VM7RRIWRIGNJGYF7IFA7JT47JN236WQK4Q, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.519444740 (348h23m30.040268486s) [noise=true , address=193.149.164.66:4976 , piece_id=UCO52WNN2OZ76ZH7EBOB7P4SUNKYVGLSBBSSUGMMCASMMSWLJAKA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.709740270 (348h23m30.230564015s) [noise=true , address=5.231.73.3:4005 , piece_id=6WV3MTC6O73KX53L7325RKUXEH5DN5H3XLONXT6SPERCZLJWKUAQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.713723140 (348h23m30.234546885s) [noise=true , address=193.149.164.69:4090 , piece_id=67PYHZH7TM7Q7RSBENSFLN3UPEN2UQ5TZVFLRZWDSPJJXSHPUQNQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:19.918545661 (348h23m30.439369406s) [noise=true , address=45.91.116.12:4237 , piece_id=IIZTIQCD6WZLVTGD4TIBQFR27FANQMRZJHDNHSFBOL5LBHIO2MBA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:11:20.161292751 (348h23m30.682116496s) [noise=true , address=145.79.227.37:4592 , piece_id=DJZEK6NZFQMBQCRCGKL7NHULLI4FGMNND3I5R55Y3GR73AGI4UOA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: <nil> May 15 07:12:16.477485165 (348h24m26.9983089s) [noise=true , address=167.17.72.68:4671 , piece_id=DTQQVGJKQDMFEI2K6QSVGOUXLOUC4KV5ZSQEYQNBMONWV43XYTMQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12NLyT9yuQVGaZYWDPRGyEeL9HhYFT9BH4hmtVjfJQeZQMbHGPc, address:167.17.72.68:4671): protocol: rpc: dial tcp 167.17.72.68:4671: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477769497 (348h24m26.998593242s) [noise=true , address=167.17.72.69:4721 , piece_id=LTCQVH4GHXUSXROKMGHTAHMMYA3NPTVNMBFAFMUI42ITHGM2YTZQ, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:1gWFQbHXFFuQMNRRpTqpM6V2R9DJmPtogsxDkgWES3pGhBddyN, address:167.17.72.69:4721): protocol: write tcp 188.166.225.170:40086->167.17.72.69:4721: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477901850 (348h24m26.998725595s) [noise=true , address=167.17.72.70:4767 , piece_id=35H75YFSGPC7EWXRH5EUZOSYSPER4MS73P6XMRQSARPY6GMMF3XA, seg_pos={0 0}, upload=333776]: Done uploading piece. err: ecclient: upload failed (node:12GyCnxNSzFpMZgnxBVVKyR6WkY1nrzqQWkuLLpwBri7kWP5rjc, address:167.17.72.70:4767): protocol: rpc: dial tcp 167.17.72.70:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 07:12:16.477969980 (348h24m26.998793725s) [seg_pos={0 0}, upload=333776]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 07:12:16.742671076 (348h24m27.263494821s) [upload=333776]: Done uploading segments... May 15 07:12:16.742703528 (348h24m27.263527273s) [upload=333776]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 7.011827716s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)