Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 1.862904364s FAIL (2025-07-29 13:34:30 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 1.862931356s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..................X.X.......................................] duration: 5.950613535s FAIL (2025-07-29 13:34:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ding segments... Jul 29 13:03:41.091320246 (20h17m53.567258701s) [upload=19386]: Got next segment. Inline: false Jul 29 13:03:41.091349191 (20h17m53.567287656s) [upload=19386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 13:03:41.091354331 (20h17m53.567292796s) [upload=19386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 13:03:41.616444845 (20h17m54.09238331s) [seg_pos={0 0}, upload=19386]: Begin upload segment... Jul 29 13:03:41.616487748 (20h17m54.092426213s) [seg_pos={0 0}, upload=19386]: Done begin upload segment. Jul 29 13:03:41.616507376 (20h17m54.092445841s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616564315 (20h17m54.09250278s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616866433 (20h17m54.092804888s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617072557 (20h17m54.093011022s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617110629 (20h17m54.093049084s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617355037 (20h17m54.093293492s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617481549 (20h17m54.093420024s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617583044 (20h17m54.093521509s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617828073 (20h17m54.093766528s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617987597 (20h17m54.093926062s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618055948 (20h17m54.093994403s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618285828 (20h17m54.094224293s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618485871 (20h17m54.094424336s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618663881 (20h17m54.094602346s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618840449 (20h17m54.094778914s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619028879 (20h17m54.094967344s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619215837 (20h17m54.095154302s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619408406 (20h17m54.095346871s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:43.299790904 (20h17m55.775729369s) [upload=19386]: Next returned nil segment Jul 29 13:03:43.299806024 (20h17m55.775744479s) [upload=19386]: Waiting for error group managing segments... Jul 29 13:03:43.847040928 (20h17m56.322979393s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:43.879495404 (20h17m56.355433869s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:46.350727192 (20h17m58.826665657s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:51.702944756 (20h18m4.178883221s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:55.250827413 (20h18m7.726765878s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:57.858469692 (20h18m10.334408157s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.265385056 (20h18m11.741323511s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.973580263 (20h18m12.449518729s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:16.002200978 (20h18m28.478139453s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:30.906580908 (20h18m43.382519373s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:31.164952501 (20h18m43.640890966s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:33.000876204 (20h18m45.476814679s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:34.725342452 (20h18m47.201280907s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:40.451024021 (20h18m52.926962486s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12B7ox4ehqxJ2mwqeXj8C9N9GuSU5etNC23iT1pJnTgMLGBAGQK, address:104.36.82.46:4500): protocol: write tcp 188.166.225.170:46700->104.36.82.46:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451060500 (20h18m52.926998965s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1eTSSYh294r2TyToU22xzVunbLwXwbaVJopvF5ej8RxaDLdbtK, address:104.36.82.51:4075): protocol: write tcp 188.166.225.170:41162->104.36.82.51:4075: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451086179 (20h18m52.927024644s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12ou7jwFyJP3mNoALddniUTDT9duVyQKZjeq4xHCyx6sRWpwEPJ, address:104.36.82.54:4175): protocol: rpc: dial tcp 104.36.82.54:4175: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451120024 (20h18m52.927058489s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1nJ8uqwxf6x4hX52AjAdCbMKi1BKGGbMvrQbiazJdLpn6j69ay, address:104.36.82.13:4214): protocol: rpc: dial tcp 104.36.82.13:4214: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451147116 (20h18m52.927085581s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1f97yBEvcrYHqMq7iovVoju77jzhUYgoQCsUGULeTagugFXrCu, address:104.36.82.15:4263): protocol: write tcp 188.166.225.170:39380->104.36.82.15:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451167805 (20h18m52.927106271s) [seg_pos={0 0}, upload=19386]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 29 13:04:40.696628655 (20h18m53.17256714s) [upload=19386]: Done uploading segments... Jul 29 13:04:40.696658391 (20h18m53.172596856s) [upload=19386]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.950638893s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.X........X...X..XX.................................X.......] duration: 5.711447426s FAIL (2025-07-29 13:33:54 UTC) orbiter warning: digest mismatch: after 786d2c4be8d438b2c616b59df6d4f93d4922ed144e0f79b484237567514766eb != before 754c097f166f520f3e7f168fcc989b028ff6249f3b26903ffdb38389ae26cd4d Duration: 5.711472774s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 1.880901612s FAIL (2025-07-29 13:33:36 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 1.880953822s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [..X........X...X..XX.................................X......] duration: 5.891250374s FAIL (2025-07-29 13:33:24 UTC) orbiter warning: digest mismatch: after 786d2c4be8d438b2c616b59df6d4f93d4922ed144e0f79b484237567514766eb != before 754c097f166f520f3e7f168fcc989b028ff6249f3b26903ffdb38389ae26cd4d Duration: 5.89127467s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...................X.X......................................] duration: 6.414601262s FAIL (2025-07-29 13:32:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ding segments... Jul 29 13:03:41.091320246 (20h17m53.567258701s) [upload=19386]: Got next segment. Inline: false Jul 29 13:03:41.091349191 (20h17m53.567287656s) [upload=19386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 13:03:41.091354331 (20h17m53.567292796s) [upload=19386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 13:03:41.616444845 (20h17m54.09238331s) [seg_pos={0 0}, upload=19386]: Begin upload segment... Jul 29 13:03:41.616487748 (20h17m54.092426213s) [seg_pos={0 0}, upload=19386]: Done begin upload segment. Jul 29 13:03:41.616507376 (20h17m54.092445841s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616564315 (20h17m54.09250278s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616866433 (20h17m54.092804888s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617072557 (20h17m54.093011022s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617110629 (20h17m54.093049084s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617355037 (20h17m54.093293492s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617481549 (20h17m54.093420024s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617583044 (20h17m54.093521509s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617828073 (20h17m54.093766528s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617987597 (20h17m54.093926062s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618055948 (20h17m54.093994403s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618285828 (20h17m54.094224293s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618485871 (20h17m54.094424336s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618663881 (20h17m54.094602346s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618840449 (20h17m54.094778914s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619028879 (20h17m54.094967344s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619215837 (20h17m54.095154302s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619408406 (20h17m54.095346871s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:43.299790904 (20h17m55.775729369s) [upload=19386]: Next returned nil segment Jul 29 13:03:43.299806024 (20h17m55.775744479s) [upload=19386]: Waiting for error group managing segments... Jul 29 13:03:43.847040928 (20h17m56.322979393s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:43.879495404 (20h17m56.355433869s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:46.350727192 (20h17m58.826665657s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:51.702944756 (20h18m4.178883221s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:55.250827413 (20h18m7.726765878s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:57.858469692 (20h18m10.334408157s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.265385056 (20h18m11.741323511s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.973580263 (20h18m12.449518729s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:16.002200978 (20h18m28.478139453s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:30.906580908 (20h18m43.382519373s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:31.164952501 (20h18m43.640890966s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:33.000876204 (20h18m45.476814679s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:34.725342452 (20h18m47.201280907s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:40.451024021 (20h18m52.926962486s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12B7ox4ehqxJ2mwqeXj8C9N9GuSU5etNC23iT1pJnTgMLGBAGQK, address:104.36.82.46:4500): protocol: write tcp 188.166.225.170:46700->104.36.82.46:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451060500 (20h18m52.926998965s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1eTSSYh294r2TyToU22xzVunbLwXwbaVJopvF5ej8RxaDLdbtK, address:104.36.82.51:4075): protocol: write tcp 188.166.225.170:41162->104.36.82.51:4075: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451086179 (20h18m52.927024644s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12ou7jwFyJP3mNoALddniUTDT9duVyQKZjeq4xHCyx6sRWpwEPJ, address:104.36.82.54:4175): protocol: rpc: dial tcp 104.36.82.54:4175: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451120024 (20h18m52.927058489s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1nJ8uqwxf6x4hX52AjAdCbMKi1BKGGbMvrQbiazJdLpn6j69ay, address:104.36.82.13:4214): protocol: rpc: dial tcp 104.36.82.13:4214: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451147116 (20h18m52.927085581s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1f97yBEvcrYHqMq7iovVoju77jzhUYgoQCsUGULeTagugFXrCu, address:104.36.82.15:4263): protocol: write tcp 188.166.225.170:39380->104.36.82.15:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451167805 (20h18m52.927106271s) [seg_pos={0 0}, upload=19386]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 29 13:04:40.696628655 (20h18m53.17256714s) [upload=19386]: Done uploading segments... Jul 29 13:04:40.696658391 (20h18m53.172596856s) [upload=19386]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.414629787s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 1.85244337s FAIL (2025-07-29 13:32:55 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 1.852473518s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [...X........X...X..XX.................................X.....] duration: 5.852224626s FAIL (2025-07-29 13:32:47 UTC) orbiter warning: digest mismatch: after 786d2c4be8d438b2c616b59df6d4f93d4922ed144e0f79b484237567514766eb != before 754c097f166f520f3e7f168fcc989b028ff6249f3b26903ffdb38389ae26cd4d Duration: 5.852243111s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [....X........X...X..XX.................................X....] duration: 5.841655705s FAIL (2025-07-29 13:32:22 UTC) orbiter warning: digest mismatch: after 786d2c4be8d438b2c616b59df6d4f93d4922ed144e0f79b484237567514766eb != before 754c097f166f520f3e7f168fcc989b028ff6249f3b26903ffdb38389ae26cd4d Duration: 5.841681704s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....X........X...X..XX.................................X...] duration: 6.119968417s FAIL (2025-07-29 13:31:53 UTC) orbiter warning: digest mismatch: after 786d2c4be8d438b2c616b59df6d4f93d4922ed144e0f79b484237567514766eb != before 754c097f166f520f3e7f168fcc989b028ff6249f3b26903ffdb38389ae26cd4d Duration: 6.120004925s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....................X.X.....................................] duration: 5.966100728s FAIL (2025-07-29 13:31:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ding segments... Jul 29 13:03:41.091320246 (20h17m53.567258701s) [upload=19386]: Got next segment. Inline: false Jul 29 13:03:41.091349191 (20h17m53.567287656s) [upload=19386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 13:03:41.091354331 (20h17m53.567292796s) [upload=19386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 13:03:41.616444845 (20h17m54.09238331s) [seg_pos={0 0}, upload=19386]: Begin upload segment... Jul 29 13:03:41.616487748 (20h17m54.092426213s) [seg_pos={0 0}, upload=19386]: Done begin upload segment. Jul 29 13:03:41.616507376 (20h17m54.092445841s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616564315 (20h17m54.09250278s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616866433 (20h17m54.092804888s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617072557 (20h17m54.093011022s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617110629 (20h17m54.093049084s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617355037 (20h17m54.093293492s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617481549 (20h17m54.093420024s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617583044 (20h17m54.093521509s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617828073 (20h17m54.093766528s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617987597 (20h17m54.093926062s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618055948 (20h17m54.093994403s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618285828 (20h17m54.094224293s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618485871 (20h17m54.094424336s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618663881 (20h17m54.094602346s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618840449 (20h17m54.094778914s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619028879 (20h17m54.094967344s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619215837 (20h17m54.095154302s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619408406 (20h17m54.095346871s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:43.299790904 (20h17m55.775729369s) [upload=19386]: Next returned nil segment Jul 29 13:03:43.299806024 (20h17m55.775744479s) [upload=19386]: Waiting for error group managing segments... Jul 29 13:03:43.847040928 (20h17m56.322979393s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:43.879495404 (20h17m56.355433869s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:46.350727192 (20h17m58.826665657s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:51.702944756 (20h18m4.178883221s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:55.250827413 (20h18m7.726765878s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:57.858469692 (20h18m10.334408157s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.265385056 (20h18m11.741323511s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.973580263 (20h18m12.449518729s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:16.002200978 (20h18m28.478139453s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:30.906580908 (20h18m43.382519373s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:31.164952501 (20h18m43.640890966s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:33.000876204 (20h18m45.476814679s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:34.725342452 (20h18m47.201280907s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:40.451024021 (20h18m52.926962486s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12B7ox4ehqxJ2mwqeXj8C9N9GuSU5etNC23iT1pJnTgMLGBAGQK, address:104.36.82.46:4500): protocol: write tcp 188.166.225.170:46700->104.36.82.46:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451060500 (20h18m52.926998965s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1eTSSYh294r2TyToU22xzVunbLwXwbaVJopvF5ej8RxaDLdbtK, address:104.36.82.51:4075): protocol: write tcp 188.166.225.170:41162->104.36.82.51:4075: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451086179 (20h18m52.927024644s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12ou7jwFyJP3mNoALddniUTDT9duVyQKZjeq4xHCyx6sRWpwEPJ, address:104.36.82.54:4175): protocol: rpc: dial tcp 104.36.82.54:4175: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451120024 (20h18m52.927058489s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1nJ8uqwxf6x4hX52AjAdCbMKi1BKGGbMvrQbiazJdLpn6j69ay, address:104.36.82.13:4214): protocol: rpc: dial tcp 104.36.82.13:4214: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451147116 (20h18m52.927085581s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1f97yBEvcrYHqMq7iovVoju77jzhUYgoQCsUGULeTagugFXrCu, address:104.36.82.15:4263): protocol: write tcp 188.166.225.170:39380->104.36.82.15:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451167805 (20h18m52.927106271s) [seg_pos={0 0}, upload=19386]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 29 13:04:40.696628655 (20h18m53.17256714s) [upload=19386]: Done uploading segments... Jul 29 13:04:40.696658391 (20h18m53.172596856s) [upload=19386]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.966130747s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 1.859904013s FAIL (2025-07-29 13:31:41 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 1.859945893s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [......X........X...X..XX.................................X..] duration: 5.855179059s FAIL (2025-07-29 13:31:15 UTC) orbiter warning: digest mismatch: after 786d2c4be8d438b2c616b59df6d4f93d4922ed144e0f79b484237567514766eb != before 754c097f166f520f3e7f168fcc989b028ff6249f3b26903ffdb38389ae26cd4d Duration: 5.855225307s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.....................X.X....................................] duration: 5.878616923s FAIL (2025-07-29 13:31:08 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ding segments... Jul 29 13:03:41.091320246 (20h17m53.567258701s) [upload=19386]: Got next segment. Inline: false Jul 29 13:03:41.091349191 (20h17m53.567287656s) [upload=19386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 13:03:41.091354331 (20h17m53.567292796s) [upload=19386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 13:03:41.616444845 (20h17m54.09238331s) [seg_pos={0 0}, upload=19386]: Begin upload segment... Jul 29 13:03:41.616487748 (20h17m54.092426213s) [seg_pos={0 0}, upload=19386]: Done begin upload segment. Jul 29 13:03:41.616507376 (20h17m54.092445841s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616564315 (20h17m54.09250278s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616866433 (20h17m54.092804888s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617072557 (20h17m54.093011022s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617110629 (20h17m54.093049084s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617355037 (20h17m54.093293492s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617481549 (20h17m54.093420024s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617583044 (20h17m54.093521509s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617828073 (20h17m54.093766528s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617987597 (20h17m54.093926062s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618055948 (20h17m54.093994403s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618285828 (20h17m54.094224293s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618485871 (20h17m54.094424336s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618663881 (20h17m54.094602346s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618840449 (20h17m54.094778914s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619028879 (20h17m54.094967344s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619215837 (20h17m54.095154302s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619408406 (20h17m54.095346871s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:43.299790904 (20h17m55.775729369s) [upload=19386]: Next returned nil segment Jul 29 13:03:43.299806024 (20h17m55.775744479s) [upload=19386]: Waiting for error group managing segments... Jul 29 13:03:43.847040928 (20h17m56.322979393s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:43.879495404 (20h17m56.355433869s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:46.350727192 (20h17m58.826665657s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:51.702944756 (20h18m4.178883221s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:55.250827413 (20h18m7.726765878s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:57.858469692 (20h18m10.334408157s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.265385056 (20h18m11.741323511s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.973580263 (20h18m12.449518729s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:16.002200978 (20h18m28.478139453s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:30.906580908 (20h18m43.382519373s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:31.164952501 (20h18m43.640890966s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:33.000876204 (20h18m45.476814679s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:34.725342452 (20h18m47.201280907s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:40.451024021 (20h18m52.926962486s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12B7ox4ehqxJ2mwqeXj8C9N9GuSU5etNC23iT1pJnTgMLGBAGQK, address:104.36.82.46:4500): protocol: write tcp 188.166.225.170:46700->104.36.82.46:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451060500 (20h18m52.926998965s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1eTSSYh294r2TyToU22xzVunbLwXwbaVJopvF5ej8RxaDLdbtK, address:104.36.82.51:4075): protocol: write tcp 188.166.225.170:41162->104.36.82.51:4075: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451086179 (20h18m52.927024644s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12ou7jwFyJP3mNoALddniUTDT9duVyQKZjeq4xHCyx6sRWpwEPJ, address:104.36.82.54:4175): protocol: rpc: dial tcp 104.36.82.54:4175: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451120024 (20h18m52.927058489s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1nJ8uqwxf6x4hX52AjAdCbMKi1BKGGbMvrQbiazJdLpn6j69ay, address:104.36.82.13:4214): protocol: rpc: dial tcp 104.36.82.13:4214: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451147116 (20h18m52.927085581s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1f97yBEvcrYHqMq7iovVoju77jzhUYgoQCsUGULeTagugFXrCu, address:104.36.82.15:4263): protocol: write tcp 188.166.225.170:39380->104.36.82.15:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451167805 (20h18m52.927106271s) [seg_pos={0 0}, upload=19386]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 29 13:04:40.696628655 (20h18m53.17256714s) [upload=19386]: Done uploading segments... Jul 29 13:04:40.696658391 (20h18m53.172596856s) [upload=19386]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.878643855s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 1.875174753s FAIL (2025-07-29 13:30:35 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 1.875198658s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [......................X.X...................................] duration: 5.900279182s FAIL (2025-07-29 13:30:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ding segments... Jul 29 13:03:41.091320246 (20h17m53.567258701s) [upload=19386]: Got next segment. Inline: false Jul 29 13:03:41.091349191 (20h17m53.567287656s) [upload=19386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 13:03:41.091354331 (20h17m53.567292796s) [upload=19386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 13:03:41.616444845 (20h17m54.09238331s) [seg_pos={0 0}, upload=19386]: Begin upload segment... Jul 29 13:03:41.616487748 (20h17m54.092426213s) [seg_pos={0 0}, upload=19386]: Done begin upload segment. Jul 29 13:03:41.616507376 (20h17m54.092445841s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616564315 (20h17m54.09250278s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616866433 (20h17m54.092804888s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617072557 (20h17m54.093011022s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617110629 (20h17m54.093049084s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617355037 (20h17m54.093293492s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617481549 (20h17m54.093420024s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617583044 (20h17m54.093521509s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617828073 (20h17m54.093766528s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617987597 (20h17m54.093926062s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618055948 (20h17m54.093994403s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618285828 (20h17m54.094224293s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618485871 (20h17m54.094424336s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618663881 (20h17m54.094602346s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618840449 (20h17m54.094778914s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619028879 (20h17m54.094967344s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619215837 (20h17m54.095154302s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619408406 (20h17m54.095346871s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:43.299790904 (20h17m55.775729369s) [upload=19386]: Next returned nil segment Jul 29 13:03:43.299806024 (20h17m55.775744479s) [upload=19386]: Waiting for error group managing segments... Jul 29 13:03:43.847040928 (20h17m56.322979393s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:43.879495404 (20h17m56.355433869s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:46.350727192 (20h17m58.826665657s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:51.702944756 (20h18m4.178883221s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:55.250827413 (20h18m7.726765878s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:57.858469692 (20h18m10.334408157s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.265385056 (20h18m11.741323511s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.973580263 (20h18m12.449518729s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:16.002200978 (20h18m28.478139453s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:30.906580908 (20h18m43.382519373s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:31.164952501 (20h18m43.640890966s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:33.000876204 (20h18m45.476814679s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:34.725342452 (20h18m47.201280907s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:40.451024021 (20h18m52.926962486s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12B7ox4ehqxJ2mwqeXj8C9N9GuSU5etNC23iT1pJnTgMLGBAGQK, address:104.36.82.46:4500): protocol: write tcp 188.166.225.170:46700->104.36.82.46:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451060500 (20h18m52.926998965s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1eTSSYh294r2TyToU22xzVunbLwXwbaVJopvF5ej8RxaDLdbtK, address:104.36.82.51:4075): protocol: write tcp 188.166.225.170:41162->104.36.82.51:4075: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451086179 (20h18m52.927024644s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12ou7jwFyJP3mNoALddniUTDT9duVyQKZjeq4xHCyx6sRWpwEPJ, address:104.36.82.54:4175): protocol: rpc: dial tcp 104.36.82.54:4175: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451120024 (20h18m52.927058489s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1nJ8uqwxf6x4hX52AjAdCbMKi1BKGGbMvrQbiazJdLpn6j69ay, address:104.36.82.13:4214): protocol: rpc: dial tcp 104.36.82.13:4214: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451147116 (20h18m52.927085581s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1f97yBEvcrYHqMq7iovVoju77jzhUYgoQCsUGULeTagugFXrCu, address:104.36.82.15:4263): protocol: write tcp 188.166.225.170:39380->104.36.82.15:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451167805 (20h18m52.927106271s) [seg_pos={0 0}, upload=19386]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 29 13:04:40.696628655 (20h18m53.17256714s) [upload=19386]: Done uploading segments... Jul 29 13:04:40.696658391 (20h18m53.172596856s) [upload=19386]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.900303437s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.......X........X...X..XX.................................X.] duration: 5.940434268s FAIL (2025-07-29 13:30:29 UTC) orbiter warning: digest mismatch: after 786d2c4be8d438b2c616b59df6d4f93d4922ed144e0f79b484237567514766eb != before 754c097f166f520f3e7f168fcc989b028ff6249f3b26903ffdb38389ae26cd4d Duration: 5.940486197s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [........X........X...X..XX.................................X] duration: 5.824626076s FAIL (2025-07-29 13:29:56 UTC) orbiter warning: digest mismatch: after 786d2c4be8d438b2c616b59df6d4f93d4922ed144e0f79b484237567514766eb != before 754c097f166f520f3e7f168fcc989b028ff6249f3b26903ffdb38389ae26cd4d Duration: 5.824647105s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.......................X.X..................................] duration: 5.697324815s FAIL (2025-07-29 13:29:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ding segments... Jul 29 13:03:41.091320246 (20h17m53.567258701s) [upload=19386]: Got next segment. Inline: false Jul 29 13:03:41.091349191 (20h17m53.567287656s) [upload=19386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 13:03:41.091354331 (20h17m53.567292796s) [upload=19386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 13:03:41.616444845 (20h17m54.09238331s) [seg_pos={0 0}, upload=19386]: Begin upload segment... Jul 29 13:03:41.616487748 (20h17m54.092426213s) [seg_pos={0 0}, upload=19386]: Done begin upload segment. Jul 29 13:03:41.616507376 (20h17m54.092445841s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616564315 (20h17m54.09250278s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616866433 (20h17m54.092804888s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617072557 (20h17m54.093011022s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617110629 (20h17m54.093049084s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617355037 (20h17m54.093293492s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617481549 (20h17m54.093420024s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617583044 (20h17m54.093521509s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617828073 (20h17m54.093766528s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617987597 (20h17m54.093926062s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618055948 (20h17m54.093994403s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618285828 (20h17m54.094224293s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618485871 (20h17m54.094424336s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618663881 (20h17m54.094602346s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618840449 (20h17m54.094778914s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619028879 (20h17m54.094967344s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619215837 (20h17m54.095154302s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619408406 (20h17m54.095346871s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:43.299790904 (20h17m55.775729369s) [upload=19386]: Next returned nil segment Jul 29 13:03:43.299806024 (20h17m55.775744479s) [upload=19386]: Waiting for error group managing segments... Jul 29 13:03:43.847040928 (20h17m56.322979393s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:43.879495404 (20h17m56.355433869s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:46.350727192 (20h17m58.826665657s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:51.702944756 (20h18m4.178883221s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:55.250827413 (20h18m7.726765878s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:57.858469692 (20h18m10.334408157s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.265385056 (20h18m11.741323511s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.973580263 (20h18m12.449518729s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:16.002200978 (20h18m28.478139453s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:30.906580908 (20h18m43.382519373s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:31.164952501 (20h18m43.640890966s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:33.000876204 (20h18m45.476814679s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:34.725342452 (20h18m47.201280907s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:40.451024021 (20h18m52.926962486s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12B7ox4ehqxJ2mwqeXj8C9N9GuSU5etNC23iT1pJnTgMLGBAGQK, address:104.36.82.46:4500): protocol: write tcp 188.166.225.170:46700->104.36.82.46:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451060500 (20h18m52.926998965s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1eTSSYh294r2TyToU22xzVunbLwXwbaVJopvF5ej8RxaDLdbtK, address:104.36.82.51:4075): protocol: write tcp 188.166.225.170:41162->104.36.82.51:4075: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451086179 (20h18m52.927024644s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12ou7jwFyJP3mNoALddniUTDT9duVyQKZjeq4xHCyx6sRWpwEPJ, address:104.36.82.54:4175): protocol: rpc: dial tcp 104.36.82.54:4175: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451120024 (20h18m52.927058489s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1nJ8uqwxf6x4hX52AjAdCbMKi1BKGGbMvrQbiazJdLpn6j69ay, address:104.36.82.13:4214): protocol: rpc: dial tcp 104.36.82.13:4214: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451147116 (20h18m52.927085581s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1f97yBEvcrYHqMq7iovVoju77jzhUYgoQCsUGULeTagugFXrCu, address:104.36.82.15:4263): protocol: write tcp 188.166.225.170:39380->104.36.82.15:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451167805 (20h18m52.927106271s) [seg_pos={0 0}, upload=19386]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 29 13:04:40.696628655 (20h18m53.17256714s) [upload=19386]: Done uploading segments... Jul 29 13:04:40.696658391 (20h18m53.172596856s) [upload=19386]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.697348242s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 1.871710081s FAIL (2025-07-29 13:29:36 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 1.871734089s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........................X.X.................................] duration: 5.678664756s FAIL (2025-07-29 13:29:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ding segments... Jul 29 13:03:41.091320246 (20h17m53.567258701s) [upload=19386]: Got next segment. Inline: false Jul 29 13:03:41.091349191 (20h17m53.567287656s) [upload=19386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 13:03:41.091354331 (20h17m53.567292796s) [upload=19386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 13:03:41.616444845 (20h17m54.09238331s) [seg_pos={0 0}, upload=19386]: Begin upload segment... Jul 29 13:03:41.616487748 (20h17m54.092426213s) [seg_pos={0 0}, upload=19386]: Done begin upload segment. Jul 29 13:03:41.616507376 (20h17m54.092445841s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616564315 (20h17m54.09250278s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616866433 (20h17m54.092804888s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617072557 (20h17m54.093011022s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617110629 (20h17m54.093049084s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617355037 (20h17m54.093293492s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617481549 (20h17m54.093420024s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617583044 (20h17m54.093521509s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617828073 (20h17m54.093766528s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617987597 (20h17m54.093926062s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618055948 (20h17m54.093994403s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618285828 (20h17m54.094224293s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618485871 (20h17m54.094424336s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618663881 (20h17m54.094602346s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618840449 (20h17m54.094778914s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619028879 (20h17m54.094967344s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619215837 (20h17m54.095154302s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619408406 (20h17m54.095346871s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:43.299790904 (20h17m55.775729369s) [upload=19386]: Next returned nil segment Jul 29 13:03:43.299806024 (20h17m55.775744479s) [upload=19386]: Waiting for error group managing segments... Jul 29 13:03:43.847040928 (20h17m56.322979393s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:43.879495404 (20h17m56.355433869s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:46.350727192 (20h17m58.826665657s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:51.702944756 (20h18m4.178883221s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:55.250827413 (20h18m7.726765878s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:57.858469692 (20h18m10.334408157s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.265385056 (20h18m11.741323511s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.973580263 (20h18m12.449518729s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:16.002200978 (20h18m28.478139453s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:30.906580908 (20h18m43.382519373s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:31.164952501 (20h18m43.640890966s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:33.000876204 (20h18m45.476814679s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:34.725342452 (20h18m47.201280907s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:40.451024021 (20h18m52.926962486s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12B7ox4ehqxJ2mwqeXj8C9N9GuSU5etNC23iT1pJnTgMLGBAGQK, address:104.36.82.46:4500): protocol: write tcp 188.166.225.170:46700->104.36.82.46:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451060500 (20h18m52.926998965s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1eTSSYh294r2TyToU22xzVunbLwXwbaVJopvF5ej8RxaDLdbtK, address:104.36.82.51:4075): protocol: write tcp 188.166.225.170:41162->104.36.82.51:4075: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451086179 (20h18m52.927024644s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12ou7jwFyJP3mNoALddniUTDT9duVyQKZjeq4xHCyx6sRWpwEPJ, address:104.36.82.54:4175): protocol: rpc: dial tcp 104.36.82.54:4175: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451120024 (20h18m52.927058489s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1nJ8uqwxf6x4hX52AjAdCbMKi1BKGGbMvrQbiazJdLpn6j69ay, address:104.36.82.13:4214): protocol: rpc: dial tcp 104.36.82.13:4214: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451147116 (20h18m52.927085581s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1f97yBEvcrYHqMq7iovVoju77jzhUYgoQCsUGULeTagugFXrCu, address:104.36.82.15:4263): protocol: write tcp 188.166.225.170:39380->104.36.82.15:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451167805 (20h18m52.927106271s) [seg_pos={0 0}, upload=19386]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 29 13:04:40.696628655 (20h18m53.17256714s) [upload=19386]: Done uploading segments... Jul 29 13:04:40.696658391 (20h18m53.172596856s) [upload=19386]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.678689133s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 1.860953963s FAIL (2025-07-29 13:29:12 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 1.86097412s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........................X.X................................] duration: 5.767199177s FAIL (2025-07-29 13:28:51 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ding segments... Jul 29 13:03:41.091320246 (20h17m53.567258701s) [upload=19386]: Got next segment. Inline: false Jul 29 13:03:41.091349191 (20h17m53.567287656s) [upload=19386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 13:03:41.091354331 (20h17m53.567292796s) [upload=19386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 13:03:41.616444845 (20h17m54.09238331s) [seg_pos={0 0}, upload=19386]: Begin upload segment... Jul 29 13:03:41.616487748 (20h17m54.092426213s) [seg_pos={0 0}, upload=19386]: Done begin upload segment. Jul 29 13:03:41.616507376 (20h17m54.092445841s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616564315 (20h17m54.09250278s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616866433 (20h17m54.092804888s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617072557 (20h17m54.093011022s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617110629 (20h17m54.093049084s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617355037 (20h17m54.093293492s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617481549 (20h17m54.093420024s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617583044 (20h17m54.093521509s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617828073 (20h17m54.093766528s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617987597 (20h17m54.093926062s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618055948 (20h17m54.093994403s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618285828 (20h17m54.094224293s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618485871 (20h17m54.094424336s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618663881 (20h17m54.094602346s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618840449 (20h17m54.094778914s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619028879 (20h17m54.094967344s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619215837 (20h17m54.095154302s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619408406 (20h17m54.095346871s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:43.299790904 (20h17m55.775729369s) [upload=19386]: Next returned nil segment Jul 29 13:03:43.299806024 (20h17m55.775744479s) [upload=19386]: Waiting for error group managing segments... Jul 29 13:03:43.847040928 (20h17m56.322979393s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:43.879495404 (20h17m56.355433869s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:46.350727192 (20h17m58.826665657s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:51.702944756 (20h18m4.178883221s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:55.250827413 (20h18m7.726765878s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:57.858469692 (20h18m10.334408157s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.265385056 (20h18m11.741323511s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.973580263 (20h18m12.449518729s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:16.002200978 (20h18m28.478139453s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:30.906580908 (20h18m43.382519373s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:31.164952501 (20h18m43.640890966s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:33.000876204 (20h18m45.476814679s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:34.725342452 (20h18m47.201280907s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:40.451024021 (20h18m52.926962486s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12B7ox4ehqxJ2mwqeXj8C9N9GuSU5etNC23iT1pJnTgMLGBAGQK, address:104.36.82.46:4500): protocol: write tcp 188.166.225.170:46700->104.36.82.46:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451060500 (20h18m52.926998965s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1eTSSYh294r2TyToU22xzVunbLwXwbaVJopvF5ej8RxaDLdbtK, address:104.36.82.51:4075): protocol: write tcp 188.166.225.170:41162->104.36.82.51:4075: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451086179 (20h18m52.927024644s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12ou7jwFyJP3mNoALddniUTDT9duVyQKZjeq4xHCyx6sRWpwEPJ, address:104.36.82.54:4175): protocol: rpc: dial tcp 104.36.82.54:4175: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451120024 (20h18m52.927058489s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1nJ8uqwxf6x4hX52AjAdCbMKi1BKGGbMvrQbiazJdLpn6j69ay, address:104.36.82.13:4214): protocol: rpc: dial tcp 104.36.82.13:4214: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451147116 (20h18m52.927085581s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1f97yBEvcrYHqMq7iovVoju77jzhUYgoQCsUGULeTagugFXrCu, address:104.36.82.15:4263): protocol: write tcp 188.166.225.170:39380->104.36.82.15:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451167805 (20h18m52.927106271s) [seg_pos={0 0}, upload=19386]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 29 13:04:40.696628655 (20h18m53.17256714s) [upload=19386]: Done uploading segments... Jul 29 13:04:40.696658391 (20h18m53.172596856s) [upload=19386]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.76722271s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.........X........X...X..XX.................................] duration: 6.096569876s FAIL (2025-07-29 13:28:50 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 7) Duration: 6.096591496s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.04307114s FAIL (2025-07-29 13:28:32 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 2.043098674s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..........X........X...X..XX................................] duration: 6.025499248s FAIL (2025-07-29 13:28:23 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 7) Duration: 6.025519076s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........................X.X...............................] duration: 5.847905186s FAIL (2025-07-29 13:28:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ding segments... Jul 29 13:03:41.091320246 (20h17m53.567258701s) [upload=19386]: Got next segment. Inline: false Jul 29 13:03:41.091349191 (20h17m53.567287656s) [upload=19386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 13:03:41.091354331 (20h17m53.567292796s) [upload=19386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 13:03:41.616444845 (20h17m54.09238331s) [seg_pos={0 0}, upload=19386]: Begin upload segment... Jul 29 13:03:41.616487748 (20h17m54.092426213s) [seg_pos={0 0}, upload=19386]: Done begin upload segment. Jul 29 13:03:41.616507376 (20h17m54.092445841s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616564315 (20h17m54.09250278s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616866433 (20h17m54.092804888s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617072557 (20h17m54.093011022s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617110629 (20h17m54.093049084s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617355037 (20h17m54.093293492s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617481549 (20h17m54.093420024s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617583044 (20h17m54.093521509s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617828073 (20h17m54.093766528s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617987597 (20h17m54.093926062s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618055948 (20h17m54.093994403s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618285828 (20h17m54.094224293s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618485871 (20h17m54.094424336s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618663881 (20h17m54.094602346s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618840449 (20h17m54.094778914s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619028879 (20h17m54.094967344s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619215837 (20h17m54.095154302s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619408406 (20h17m54.095346871s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:43.299790904 (20h17m55.775729369s) [upload=19386]: Next returned nil segment Jul 29 13:03:43.299806024 (20h17m55.775744479s) [upload=19386]: Waiting for error group managing segments... Jul 29 13:03:43.847040928 (20h17m56.322979393s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:43.879495404 (20h17m56.355433869s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:46.350727192 (20h17m58.826665657s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:51.702944756 (20h18m4.178883221s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:55.250827413 (20h18m7.726765878s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:57.858469692 (20h18m10.334408157s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.265385056 (20h18m11.741323511s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.973580263 (20h18m12.449518729s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:16.002200978 (20h18m28.478139453s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:30.906580908 (20h18m43.382519373s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:31.164952501 (20h18m43.640890966s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:33.000876204 (20h18m45.476814679s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:34.725342452 (20h18m47.201280907s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:40.451024021 (20h18m52.926962486s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12B7ox4ehqxJ2mwqeXj8C9N9GuSU5etNC23iT1pJnTgMLGBAGQK, address:104.36.82.46:4500): protocol: write tcp 188.166.225.170:46700->104.36.82.46:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451060500 (20h18m52.926998965s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1eTSSYh294r2TyToU22xzVunbLwXwbaVJopvF5ej8RxaDLdbtK, address:104.36.82.51:4075): protocol: write tcp 188.166.225.170:41162->104.36.82.51:4075: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451086179 (20h18m52.927024644s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12ou7jwFyJP3mNoALddniUTDT9duVyQKZjeq4xHCyx6sRWpwEPJ, address:104.36.82.54:4175): protocol: rpc: dial tcp 104.36.82.54:4175: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451120024 (20h18m52.927058489s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1nJ8uqwxf6x4hX52AjAdCbMKi1BKGGbMvrQbiazJdLpn6j69ay, address:104.36.82.13:4214): protocol: rpc: dial tcp 104.36.82.13:4214: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451147116 (20h18m52.927085581s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1f97yBEvcrYHqMq7iovVoju77jzhUYgoQCsUGULeTagugFXrCu, address:104.36.82.15:4263): protocol: write tcp 188.166.225.170:39380->104.36.82.15:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451167805 (20h18m52.927106271s) [seg_pos={0 0}, upload=19386]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 29 13:04:40.696628655 (20h18m53.17256714s) [upload=19386]: Done uploading segments... Jul 29 13:04:40.696658391 (20h18m53.172596856s) [upload=19386]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.847929483s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...........X........X...X..XX...............................] duration: 6.091377302s FAIL (2025-07-29 13:28:02 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 7) Duration: 6.09140764s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [X...........X...............................................] duration: 2.224986963s FAIL (2025-07-29 13:27:31 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 2.225013485s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...........................X.X..............................] duration: 6.00720901s FAIL (2025-07-29 13:27:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ding segments... Jul 29 13:03:41.091320246 (20h17m53.567258701s) [upload=19386]: Got next segment. Inline: false Jul 29 13:03:41.091349191 (20h17m53.567287656s) [upload=19386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 13:03:41.091354331 (20h17m53.567292796s) [upload=19386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 13:03:41.616444845 (20h17m54.09238331s) [seg_pos={0 0}, upload=19386]: Begin upload segment... Jul 29 13:03:41.616487748 (20h17m54.092426213s) [seg_pos={0 0}, upload=19386]: Done begin upload segment. Jul 29 13:03:41.616507376 (20h17m54.092445841s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616564315 (20h17m54.09250278s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616866433 (20h17m54.092804888s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617072557 (20h17m54.093011022s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617110629 (20h17m54.093049084s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617355037 (20h17m54.093293492s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617481549 (20h17m54.093420024s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617583044 (20h17m54.093521509s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617828073 (20h17m54.093766528s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617987597 (20h17m54.093926062s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618055948 (20h17m54.093994403s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618285828 (20h17m54.094224293s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618485871 (20h17m54.094424336s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618663881 (20h17m54.094602346s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618840449 (20h17m54.094778914s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619028879 (20h17m54.094967344s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619215837 (20h17m54.095154302s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619408406 (20h17m54.095346871s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:43.299790904 (20h17m55.775729369s) [upload=19386]: Next returned nil segment Jul 29 13:03:43.299806024 (20h17m55.775744479s) [upload=19386]: Waiting for error group managing segments... Jul 29 13:03:43.847040928 (20h17m56.322979393s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:43.879495404 (20h17m56.355433869s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:46.350727192 (20h17m58.826665657s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:51.702944756 (20h18m4.178883221s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:55.250827413 (20h18m7.726765878s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:57.858469692 (20h18m10.334408157s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.265385056 (20h18m11.741323511s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.973580263 (20h18m12.449518729s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:16.002200978 (20h18m28.478139453s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:30.906580908 (20h18m43.382519373s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:31.164952501 (20h18m43.640890966s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:33.000876204 (20h18m45.476814679s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:34.725342452 (20h18m47.201280907s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:40.451024021 (20h18m52.926962486s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12B7ox4ehqxJ2mwqeXj8C9N9GuSU5etNC23iT1pJnTgMLGBAGQK, address:104.36.82.46:4500): protocol: write tcp 188.166.225.170:46700->104.36.82.46:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451060500 (20h18m52.926998965s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1eTSSYh294r2TyToU22xzVunbLwXwbaVJopvF5ej8RxaDLdbtK, address:104.36.82.51:4075): protocol: write tcp 188.166.225.170:41162->104.36.82.51:4075: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451086179 (20h18m52.927024644s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12ou7jwFyJP3mNoALddniUTDT9duVyQKZjeq4xHCyx6sRWpwEPJ, address:104.36.82.54:4175): protocol: rpc: dial tcp 104.36.82.54:4175: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451120024 (20h18m52.927058489s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1nJ8uqwxf6x4hX52AjAdCbMKi1BKGGbMvrQbiazJdLpn6j69ay, address:104.36.82.13:4214): protocol: rpc: dial tcp 104.36.82.13:4214: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451147116 (20h18m52.927085581s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1f97yBEvcrYHqMq7iovVoju77jzhUYgoQCsUGULeTagugFXrCu, address:104.36.82.15:4263): protocol: write tcp 188.166.225.170:39380->104.36.82.15:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451167805 (20h18m52.927106271s) [seg_pos={0 0}, upload=19386]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 29 13:04:40.696628655 (20h18m53.17256714s) [upload=19386]: Done uploading segments... Jul 29 13:04:40.696658391 (20h18m53.172596856s) [upload=19386]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.007235201s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [............................X.X.............................] duration: 5.97225954s FAIL (2025-07-29 13:26:58 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ding segments... Jul 29 13:03:41.091320246 (20h17m53.567258701s) [upload=19386]: Got next segment. Inline: false Jul 29 13:03:41.091349191 (20h17m53.567287656s) [upload=19386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 13:03:41.091354331 (20h17m53.567292796s) [upload=19386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 13:03:41.616444845 (20h17m54.09238331s) [seg_pos={0 0}, upload=19386]: Begin upload segment... Jul 29 13:03:41.616487748 (20h17m54.092426213s) [seg_pos={0 0}, upload=19386]: Done begin upload segment. Jul 29 13:03:41.616507376 (20h17m54.092445841s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616564315 (20h17m54.09250278s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616866433 (20h17m54.092804888s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617072557 (20h17m54.093011022s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617110629 (20h17m54.093049084s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617355037 (20h17m54.093293492s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617481549 (20h17m54.093420024s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617583044 (20h17m54.093521509s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617828073 (20h17m54.093766528s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617987597 (20h17m54.093926062s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618055948 (20h17m54.093994403s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618285828 (20h17m54.094224293s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618485871 (20h17m54.094424336s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618663881 (20h17m54.094602346s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618840449 (20h17m54.094778914s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619028879 (20h17m54.094967344s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619215837 (20h17m54.095154302s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619408406 (20h17m54.095346871s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:43.299790904 (20h17m55.775729369s) [upload=19386]: Next returned nil segment Jul 29 13:03:43.299806024 (20h17m55.775744479s) [upload=19386]: Waiting for error group managing segments... Jul 29 13:03:43.847040928 (20h17m56.322979393s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:43.879495404 (20h17m56.355433869s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:46.350727192 (20h17m58.826665657s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:51.702944756 (20h18m4.178883221s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:55.250827413 (20h18m7.726765878s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:57.858469692 (20h18m10.334408157s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.265385056 (20h18m11.741323511s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.973580263 (20h18m12.449518729s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:16.002200978 (20h18m28.478139453s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:30.906580908 (20h18m43.382519373s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:31.164952501 (20h18m43.640890966s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:33.000876204 (20h18m45.476814679s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:34.725342452 (20h18m47.201280907s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:40.451024021 (20h18m52.926962486s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12B7ox4ehqxJ2mwqeXj8C9N9GuSU5etNC23iT1pJnTgMLGBAGQK, address:104.36.82.46:4500): protocol: write tcp 188.166.225.170:46700->104.36.82.46:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451060500 (20h18m52.926998965s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1eTSSYh294r2TyToU22xzVunbLwXwbaVJopvF5ej8RxaDLdbtK, address:104.36.82.51:4075): protocol: write tcp 188.166.225.170:41162->104.36.82.51:4075: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451086179 (20h18m52.927024644s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12ou7jwFyJP3mNoALddniUTDT9duVyQKZjeq4xHCyx6sRWpwEPJ, address:104.36.82.54:4175): protocol: rpc: dial tcp 104.36.82.54:4175: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451120024 (20h18m52.927058489s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1nJ8uqwxf6x4hX52AjAdCbMKi1BKGGbMvrQbiazJdLpn6j69ay, address:104.36.82.13:4214): protocol: rpc: dial tcp 104.36.82.13:4214: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451147116 (20h18m52.927085581s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1f97yBEvcrYHqMq7iovVoju77jzhUYgoQCsUGULeTagugFXrCu, address:104.36.82.15:4263): protocol: write tcp 188.166.225.170:39380->104.36.82.15:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451167805 (20h18m52.927106271s) [seg_pos={0 0}, upload=19386]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 29 13:04:40.696628655 (20h18m53.17256714s) [upload=19386]: Done uploading segments... Jul 29 13:04:40.696658391 (20h18m53.172596856s) [upload=19386]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.972284917s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.X...........X..............................................] duration: 1.830027892s FAIL (2025-07-29 13:26:57 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 1.830053419s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [............X........X...X..XX..............................] duration: 5.852201514s FAIL (2025-07-29 13:26:47 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 7) Duration: 5.852224297s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.............X........X...X..XX.............................] duration: 5.90874393s FAIL (2025-07-29 13:26:09 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 7) Duration: 5.908769488s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..............X........X...X..XX............................] duration: 5.93998826s FAIL (2025-07-29 13:25:52 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 7) Duration: 5.940013837s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..X...........X.............................................] duration: 2.033011882s FAIL (2025-07-29 13:25:46 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 2.033043903s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.............................X.X............................] duration: 5.875392919s FAIL (2025-07-29 13:25:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ding segments... Jul 29 13:03:41.091320246 (20h17m53.567258701s) [upload=19386]: Got next segment. Inline: false Jul 29 13:03:41.091349191 (20h17m53.567287656s) [upload=19386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 13:03:41.091354331 (20h17m53.567292796s) [upload=19386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 13:03:41.616444845 (20h17m54.09238331s) [seg_pos={0 0}, upload=19386]: Begin upload segment... Jul 29 13:03:41.616487748 (20h17m54.092426213s) [seg_pos={0 0}, upload=19386]: Done begin upload segment. Jul 29 13:03:41.616507376 (20h17m54.092445841s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616564315 (20h17m54.09250278s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616866433 (20h17m54.092804888s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617072557 (20h17m54.093011022s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617110629 (20h17m54.093049084s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617355037 (20h17m54.093293492s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617481549 (20h17m54.093420024s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617583044 (20h17m54.093521509s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617828073 (20h17m54.093766528s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617987597 (20h17m54.093926062s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618055948 (20h17m54.093994403s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618285828 (20h17m54.094224293s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618485871 (20h17m54.094424336s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618663881 (20h17m54.094602346s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618840449 (20h17m54.094778914s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619028879 (20h17m54.094967344s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619215837 (20h17m54.095154302s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619408406 (20h17m54.095346871s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:43.299790904 (20h17m55.775729369s) [upload=19386]: Next returned nil segment Jul 29 13:03:43.299806024 (20h17m55.775744479s) [upload=19386]: Waiting for error group managing segments... Jul 29 13:03:43.847040928 (20h17m56.322979393s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:43.879495404 (20h17m56.355433869s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:46.350727192 (20h17m58.826665657s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:51.702944756 (20h18m4.178883221s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:55.250827413 (20h18m7.726765878s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:57.858469692 (20h18m10.334408157s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.265385056 (20h18m11.741323511s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.973580263 (20h18m12.449518729s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:16.002200978 (20h18m28.478139453s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:30.906580908 (20h18m43.382519373s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:31.164952501 (20h18m43.640890966s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:33.000876204 (20h18m45.476814679s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:34.725342452 (20h18m47.201280907s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:40.451024021 (20h18m52.926962486s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12B7ox4ehqxJ2mwqeXj8C9N9GuSU5etNC23iT1pJnTgMLGBAGQK, address:104.36.82.46:4500): protocol: write tcp 188.166.225.170:46700->104.36.82.46:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451060500 (20h18m52.926998965s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1eTSSYh294r2TyToU22xzVunbLwXwbaVJopvF5ej8RxaDLdbtK, address:104.36.82.51:4075): protocol: write tcp 188.166.225.170:41162->104.36.82.51:4075: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451086179 (20h18m52.927024644s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12ou7jwFyJP3mNoALddniUTDT9duVyQKZjeq4xHCyx6sRWpwEPJ, address:104.36.82.54:4175): protocol: rpc: dial tcp 104.36.82.54:4175: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451120024 (20h18m52.927058489s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1nJ8uqwxf6x4hX52AjAdCbMKi1BKGGbMvrQbiazJdLpn6j69ay, address:104.36.82.13:4214): protocol: rpc: dial tcp 104.36.82.13:4214: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451147116 (20h18m52.927085581s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1f97yBEvcrYHqMq7iovVoju77jzhUYgoQCsUGULeTagugFXrCu, address:104.36.82.15:4263): protocol: write tcp 188.166.225.170:39380->104.36.82.15:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451167805 (20h18m52.927106271s) [seg_pos={0 0}, upload=19386]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 29 13:04:40.696628655 (20h18m53.17256714s) [upload=19386]: Done uploading segments... Jul 29 13:04:40.696658391 (20h18m53.172596856s) [upload=19386]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.875417755s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..............................X.X...........................] duration: 5.880098071s FAIL (2025-07-29 13:25:13 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ding segments... Jul 29 13:03:41.091320246 (20h17m53.567258701s) [upload=19386]: Got next segment. Inline: false Jul 29 13:03:41.091349191 (20h17m53.567287656s) [upload=19386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 13:03:41.091354331 (20h17m53.567292796s) [upload=19386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 13:03:41.616444845 (20h17m54.09238331s) [seg_pos={0 0}, upload=19386]: Begin upload segment... Jul 29 13:03:41.616487748 (20h17m54.092426213s) [seg_pos={0 0}, upload=19386]: Done begin upload segment. Jul 29 13:03:41.616507376 (20h17m54.092445841s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616564315 (20h17m54.09250278s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616866433 (20h17m54.092804888s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617072557 (20h17m54.093011022s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617110629 (20h17m54.093049084s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617355037 (20h17m54.093293492s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617481549 (20h17m54.093420024s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617583044 (20h17m54.093521509s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617828073 (20h17m54.093766528s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617987597 (20h17m54.093926062s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618055948 (20h17m54.093994403s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618285828 (20h17m54.094224293s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618485871 (20h17m54.094424336s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618663881 (20h17m54.094602346s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618840449 (20h17m54.094778914s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619028879 (20h17m54.094967344s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619215837 (20h17m54.095154302s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619408406 (20h17m54.095346871s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:43.299790904 (20h17m55.775729369s) [upload=19386]: Next returned nil segment Jul 29 13:03:43.299806024 (20h17m55.775744479s) [upload=19386]: Waiting for error group managing segments... Jul 29 13:03:43.847040928 (20h17m56.322979393s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:43.879495404 (20h17m56.355433869s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:46.350727192 (20h17m58.826665657s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:51.702944756 (20h18m4.178883221s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:55.250827413 (20h18m7.726765878s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:57.858469692 (20h18m10.334408157s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.265385056 (20h18m11.741323511s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.973580263 (20h18m12.449518729s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:16.002200978 (20h18m28.478139453s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:30.906580908 (20h18m43.382519373s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:31.164952501 (20h18m43.640890966s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:33.000876204 (20h18m45.476814679s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:34.725342452 (20h18m47.201280907s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:40.451024021 (20h18m52.926962486s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12B7ox4ehqxJ2mwqeXj8C9N9GuSU5etNC23iT1pJnTgMLGBAGQK, address:104.36.82.46:4500): protocol: write tcp 188.166.225.170:46700->104.36.82.46:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451060500 (20h18m52.926998965s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1eTSSYh294r2TyToU22xzVunbLwXwbaVJopvF5ej8RxaDLdbtK, address:104.36.82.51:4075): protocol: write tcp 188.166.225.170:41162->104.36.82.51:4075: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451086179 (20h18m52.927024644s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12ou7jwFyJP3mNoALddniUTDT9duVyQKZjeq4xHCyx6sRWpwEPJ, address:104.36.82.54:4175): protocol: rpc: dial tcp 104.36.82.54:4175: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451120024 (20h18m52.927058489s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1nJ8uqwxf6x4hX52AjAdCbMKi1BKGGbMvrQbiazJdLpn6j69ay, address:104.36.82.13:4214): protocol: rpc: dial tcp 104.36.82.13:4214: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451147116 (20h18m52.927085581s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1f97yBEvcrYHqMq7iovVoju77jzhUYgoQCsUGULeTagugFXrCu, address:104.36.82.15:4263): protocol: write tcp 188.166.225.170:39380->104.36.82.15:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451167805 (20h18m52.927106271s) [seg_pos={0 0}, upload=19386]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 29 13:04:40.696628655 (20h18m53.17256714s) [upload=19386]: Done uploading segments... Jul 29 13:04:40.696658391 (20h18m53.172596856s) [upload=19386]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.88012844s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...X...........X............................................] duration: 1.818130583s FAIL (2025-07-29 13:24:58 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 1.818156673s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...............X........X...X..XX...........................] duration: 5.777043487s FAIL (2025-07-29 13:24:54 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 7) Duration: 5.777063786s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [....X...........X...........................................] duration: 1.975570038s FAIL (2025-07-29 13:24:36 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 1.975599234s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...............................X.X..........................] duration: 5.864267141s FAIL (2025-07-29 13:24:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ding segments... Jul 29 13:03:41.091320246 (20h17m53.567258701s) [upload=19386]: Got next segment. Inline: false Jul 29 13:03:41.091349191 (20h17m53.567287656s) [upload=19386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 13:03:41.091354331 (20h17m53.567292796s) [upload=19386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 13:03:41.616444845 (20h17m54.09238331s) [seg_pos={0 0}, upload=19386]: Begin upload segment... Jul 29 13:03:41.616487748 (20h17m54.092426213s) [seg_pos={0 0}, upload=19386]: Done begin upload segment. Jul 29 13:03:41.616507376 (20h17m54.092445841s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616564315 (20h17m54.09250278s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616866433 (20h17m54.092804888s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617072557 (20h17m54.093011022s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617110629 (20h17m54.093049084s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617355037 (20h17m54.093293492s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617481549 (20h17m54.093420024s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617583044 (20h17m54.093521509s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617828073 (20h17m54.093766528s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617987597 (20h17m54.093926062s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618055948 (20h17m54.093994403s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618285828 (20h17m54.094224293s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618485871 (20h17m54.094424336s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618663881 (20h17m54.094602346s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618840449 (20h17m54.094778914s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619028879 (20h17m54.094967344s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619215837 (20h17m54.095154302s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619408406 (20h17m54.095346871s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:43.299790904 (20h17m55.775729369s) [upload=19386]: Next returned nil segment Jul 29 13:03:43.299806024 (20h17m55.775744479s) [upload=19386]: Waiting for error group managing segments... Jul 29 13:03:43.847040928 (20h17m56.322979393s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:43.879495404 (20h17m56.355433869s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:46.350727192 (20h17m58.826665657s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:51.702944756 (20h18m4.178883221s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:55.250827413 (20h18m7.726765878s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:57.858469692 (20h18m10.334408157s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.265385056 (20h18m11.741323511s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.973580263 (20h18m12.449518729s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:16.002200978 (20h18m28.478139453s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:30.906580908 (20h18m43.382519373s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:31.164952501 (20h18m43.640890966s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:33.000876204 (20h18m45.476814679s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:34.725342452 (20h18m47.201280907s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:40.451024021 (20h18m52.926962486s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12B7ox4ehqxJ2mwqeXj8C9N9GuSU5etNC23iT1pJnTgMLGBAGQK, address:104.36.82.46:4500): protocol: write tcp 188.166.225.170:46700->104.36.82.46:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451060500 (20h18m52.926998965s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1eTSSYh294r2TyToU22xzVunbLwXwbaVJopvF5ej8RxaDLdbtK, address:104.36.82.51:4075): protocol: write tcp 188.166.225.170:41162->104.36.82.51:4075: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451086179 (20h18m52.927024644s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12ou7jwFyJP3mNoALddniUTDT9duVyQKZjeq4xHCyx6sRWpwEPJ, address:104.36.82.54:4175): protocol: rpc: dial tcp 104.36.82.54:4175: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451120024 (20h18m52.927058489s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1nJ8uqwxf6x4hX52AjAdCbMKi1BKGGbMvrQbiazJdLpn6j69ay, address:104.36.82.13:4214): protocol: rpc: dial tcp 104.36.82.13:4214: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451147116 (20h18m52.927085581s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1f97yBEvcrYHqMq7iovVoju77jzhUYgoQCsUGULeTagugFXrCu, address:104.36.82.15:4263): protocol: write tcp 188.166.225.170:39380->104.36.82.15:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451167805 (20h18m52.927106271s) [seg_pos={0 0}, upload=19386]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 29 13:04:40.696628655 (20h18m53.17256714s) [upload=19386]: Done uploading segments... Jul 29 13:04:40.696658391 (20h18m53.172596856s) [upload=19386]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.864295314s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [X...............X........X...X..XX..........................] duration: 5.657545366s FAIL (2025-07-29 13:23:57 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 7) Duration: 5.657574003s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....X...........X..........................................] duration: 1.837296997s FAIL (2025-07-29 13:23:41 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 1.837323858s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [................................X.X.........................] duration: 5.821658829s FAIL (2025-07-29 13:23:19 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ding segments... Jul 29 13:03:41.091320246 (20h17m53.567258701s) [upload=19386]: Got next segment. Inline: false Jul 29 13:03:41.091349191 (20h17m53.567287656s) [upload=19386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 13:03:41.091354331 (20h17m53.567292796s) [upload=19386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 13:03:41.616444845 (20h17m54.09238331s) [seg_pos={0 0}, upload=19386]: Begin upload segment... Jul 29 13:03:41.616487748 (20h17m54.092426213s) [seg_pos={0 0}, upload=19386]: Done begin upload segment. Jul 29 13:03:41.616507376 (20h17m54.092445841s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616564315 (20h17m54.09250278s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616866433 (20h17m54.092804888s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617072557 (20h17m54.093011022s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617110629 (20h17m54.093049084s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617355037 (20h17m54.093293492s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617481549 (20h17m54.093420024s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617583044 (20h17m54.093521509s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617828073 (20h17m54.093766528s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617987597 (20h17m54.093926062s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618055948 (20h17m54.093994403s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618285828 (20h17m54.094224293s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618485871 (20h17m54.094424336s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618663881 (20h17m54.094602346s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618840449 (20h17m54.094778914s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619028879 (20h17m54.094967344s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619215837 (20h17m54.095154302s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619408406 (20h17m54.095346871s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:43.299790904 (20h17m55.775729369s) [upload=19386]: Next returned nil segment Jul 29 13:03:43.299806024 (20h17m55.775744479s) [upload=19386]: Waiting for error group managing segments... Jul 29 13:03:43.847040928 (20h17m56.322979393s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:43.879495404 (20h17m56.355433869s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:46.350727192 (20h17m58.826665657s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:51.702944756 (20h18m4.178883221s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:55.250827413 (20h18m7.726765878s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:57.858469692 (20h18m10.334408157s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.265385056 (20h18m11.741323511s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.973580263 (20h18m12.449518729s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:16.002200978 (20h18m28.478139453s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:30.906580908 (20h18m43.382519373s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:31.164952501 (20h18m43.640890966s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:33.000876204 (20h18m45.476814679s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:34.725342452 (20h18m47.201280907s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:40.451024021 (20h18m52.926962486s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12B7ox4ehqxJ2mwqeXj8C9N9GuSU5etNC23iT1pJnTgMLGBAGQK, address:104.36.82.46:4500): protocol: write tcp 188.166.225.170:46700->104.36.82.46:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451060500 (20h18m52.926998965s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1eTSSYh294r2TyToU22xzVunbLwXwbaVJopvF5ej8RxaDLdbtK, address:104.36.82.51:4075): protocol: write tcp 188.166.225.170:41162->104.36.82.51:4075: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451086179 (20h18m52.927024644s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12ou7jwFyJP3mNoALddniUTDT9duVyQKZjeq4xHCyx6sRWpwEPJ, address:104.36.82.54:4175): protocol: rpc: dial tcp 104.36.82.54:4175: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451120024 (20h18m52.927058489s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1nJ8uqwxf6x4hX52AjAdCbMKi1BKGGbMvrQbiazJdLpn6j69ay, address:104.36.82.13:4214): protocol: rpc: dial tcp 104.36.82.13:4214: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451147116 (20h18m52.927085581s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1f97yBEvcrYHqMq7iovVoju77jzhUYgoQCsUGULeTagugFXrCu, address:104.36.82.15:4263): protocol: write tcp 188.166.225.170:39380->104.36.82.15:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451167805 (20h18m52.927106271s) [seg_pos={0 0}, upload=19386]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 29 13:04:40.696628655 (20h18m53.17256714s) [upload=19386]: Done uploading segments... Jul 29 13:04:40.696658391 (20h18m53.172596856s) [upload=19386]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.821701862s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [......X...........X.........................................] duration: 1.862007887s FAIL (2025-07-29 13:23:09 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 1.862038977s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.X...............X........X...X..XX.........................] duration: 5.997350816s FAIL (2025-07-29 13:23:05 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 7) Duration: 5.997377755s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.................................X.X........................] duration: 6.041772689s FAIL (2025-07-29 13:22:46 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ding segments... Jul 29 13:03:41.091320246 (20h17m53.567258701s) [upload=19386]: Got next segment. Inline: false Jul 29 13:03:41.091349191 (20h17m53.567287656s) [upload=19386]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 13:03:41.091354331 (20h17m53.567292796s) [upload=19386]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 13:03:41.616444845 (20h17m54.09238331s) [seg_pos={0 0}, upload=19386]: Begin upload segment... Jul 29 13:03:41.616487748 (20h17m54.092426213s) [seg_pos={0 0}, upload=19386]: Done begin upload segment. Jul 29 13:03:41.616507376 (20h17m54.092445841s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616564315 (20h17m54.09250278s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.616866433 (20h17m54.092804888s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617072557 (20h17m54.093011022s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617110629 (20h17m54.093049084s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617355037 (20h17m54.093293492s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617481549 (20h17m54.093420024s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617583044 (20h17m54.093521509s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617828073 (20h17m54.093766528s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.617987597 (20h17m54.093926062s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618055948 (20h17m54.093994403s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618285828 (20h17m54.094224293s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618485871 (20h17m54.094424336s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618663881 (20h17m54.094602346s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.618840449 (20h17m54.094778914s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619028879 (20h17m54.094967344s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619215837 (20h17m54.095154302s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:41.619408406 (20h17m54.095346871s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Uploading piece... Jul 29 13:03:43.299790904 (20h17m55.775729369s) [upload=19386]: Next returned nil segment Jul 29 13:03:43.299806024 (20h17m55.775744479s) [upload=19386]: Waiting for error group managing segments... Jul 29 13:03:43.847040928 (20h17m56.322979393s) [noise=true , address=104.36.82.9:4028 , piece_id=XCRNGUPBFYCZOYT23L6NHJ6WQR55TQMTWZF4SNKSRLQGC5TYO3LA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:43.879495404 (20h17m56.355433869s) [noise=true , address=104.36.82.61:4391 , piece_id=TI5YDGDSTWKDX5XAWL36F3TJDQNHKNTREZVTUGIS2EVZYP6DWSDA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:46.350727192 (20h17m58.826665657s) [noise=true , address=104.36.82.5:4866 , piece_id=EFRLC4LA232NZZJU3GDSP2EEBZO3W2LGJRD3L2A6SPSRBTLUTIFA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:51.702944756 (20h18m4.178883221s) [noise=true , address=104.36.82.42:4802 , piece_id=6HORHYYKIKDX4IWWKUQPVGZQZ2YYIGC47OCZHRVBC4BZFRJX5TQQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:55.250827413 (20h18m7.726765878s) [noise=true , address=104.36.82.55:4219 , piece_id=IURHWONYUYYJWIL5BDF6HLCEPA76O6QAZAJ22NCI6LQYIGO6K4IA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:57.858469692 (20h18m10.334408157s) [noise=true , address=104.36.82.53:4126 , piece_id=GZ5Z7AGMYVGYVUJAWFAIS6SQLRHFB772RLQY736QUGOXA5OWQ34Q, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.265385056 (20h18m11.741323511s) [noise=true , address=104.36.82.10:4088 , piece_id=6IV2Z7FXDNEOBJMNPRG7WSPZWZTJ5CGDF6JHBSF7WCIIL2QQWFUA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:03:59.973580263 (20h18m12.449518729s) [noise=true , address=104.36.82.11:4119 , piece_id=6KWLABSUMX6YVKD6LGKR6ONLQKFFTSTKFHHRVFIN7U4KDALBNCBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:16.002200978 (20h18m28.478139453s) [noise=true , address=104.36.82.52:4092 , piece_id=BMJCAQYXJCUVJTEXH52UXYLD4MHWLDGQTJIPDOFRF4LPLTT36ESA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:30.906580908 (20h18m43.382519373s) [noise=true , address=104.36.82.20:4448 , piece_id=CSHYRDGTS4ROFDTKETIZ36TWJE3OEMAKLB4RGFAU557G725EANXQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:31.164952501 (20h18m43.640890966s) [noise=true , address=104.36.82.14:4227 , piece_id=O2FXJBOV3CJCKLZHRPBTLXX2QRT2OWJQ5HPNHZI5IVIFDVUUICVQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:33.000876204 (20h18m45.476814679s) [noise=true , address=104.36.82.56:4242 , piece_id=BHKC5MH4G65BK2YLJJANIYZ4MLAUTMBG32XSRISARHCO26AZUXRA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:34.725342452 (20h18m47.201280907s) [noise=true , address=104.36.82.47:4545 , piece_id=PZWVIOWLBZ72IOFRJHPAD4OZZPA2G4XYCDLGYEY7THSOR7XDHMUQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: <nil> Jul 29 13:04:40.451024021 (20h18m52.926962486s) [noise=true , address=104.36.82.46:4500 , piece_id=VCYKKDSLEKTTL3KO2FJ4VA5JGIVPA4GFBBHH65HPFZTP4REUTHYQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12B7ox4ehqxJ2mwqeXj8C9N9GuSU5etNC23iT1pJnTgMLGBAGQK, address:104.36.82.46:4500): protocol: write tcp 188.166.225.170:46700->104.36.82.46:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451060500 (20h18m52.926998965s) [noise=true , address=104.36.82.51:4075 , piece_id=P5NU32M3V7AMYFZVJ6XZY5K2YMQWF4EULQGX6RRYWG6SWDWWAFMQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1eTSSYh294r2TyToU22xzVunbLwXwbaVJopvF5ej8RxaDLdbtK, address:104.36.82.51:4075): protocol: write tcp 188.166.225.170:41162->104.36.82.51:4075: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451086179 (20h18m52.927024644s) [noise=true , address=104.36.82.54:4175 , piece_id=5PAE3BM5AFSLZ5TVUQC3QK3RVWCVGFDLOK452BDPWQ7ARTL7EYWQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:12ou7jwFyJP3mNoALddniUTDT9duVyQKZjeq4xHCyx6sRWpwEPJ, address:104.36.82.54:4175): protocol: rpc: dial tcp 104.36.82.54:4175: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451120024 (20h18m52.927058489s) [noise=true , address=104.36.82.13:4214 , piece_id=SLX3YJCEWCM3553VZHILH337FKSPPFNKVFTPXO4KIBOS6IXCLXHA, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1nJ8uqwxf6x4hX52AjAdCbMKi1BKGGbMvrQbiazJdLpn6j69ay, address:104.36.82.13:4214): protocol: rpc: dial tcp 104.36.82.13:4214: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451147116 (20h18m52.927085581s) [noise=true , address=104.36.82.15:4263 , piece_id=3GCLYN4UBCLM5BPLXDQT5SSWPJBTKIOZM4UIUNO3OZIXW6OANUBQ, seg_pos={0 0}, upload=19386]: Done uploading piece. err: ecclient: upload failed (node:1f97yBEvcrYHqMq7iovVoju77jzhUYgoQCsUGULeTagugFXrCu, address:104.36.82.15:4263): protocol: write tcp 188.166.225.170:39380->104.36.82.15:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 29 13:04:40.451167805 (20h18m52.927106271s) [seg_pos={0 0}, upload=19386]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 29 13:04:40.696628655 (20h18m53.17256714s) [upload=19386]: Done uploading segments... Jul 29 13:04:40.696658391 (20h18m53.172596856s) [upload=19386]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.041794771s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......X...........X........................................] duration: 1.997092238s FAIL (2025-07-29 13:22:33 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 1.997118639s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [..X...............X........X...X..XX........................] duration: 6.000204311s FAIL (2025-07-29 13:22:15 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 14 but got 7) Duration: 6.000246761s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)