Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 3.430503258s FAIL (2025-10-27 20:26:44 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.430528024s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 4.963781425s FAIL (2025-10-27 20:26:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== eg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577374904 (5h14m15.926682211s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577388009 (5h14m15.926695285s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577699713 (5h14m15.927006989s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577834229 (5h14m15.927141505s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578014502 (5h14m15.927321778s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578280880 (5h14m15.927588156s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578608904 (5h14m15.92791618s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578935426 (5h14m15.928242712s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579041648 (5h14m15.928348924s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579222723 (5h14m15.928529999s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579486625 (5h14m15.928793901s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579780545 (5h14m15.929087821s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.580102097 (5h14m15.929409403s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:59.402940250 (5h14m16.752247547s) [upload=4621]: Next returned nil segment Oct 27 19:56:59.402970929 (5h14m16.752278235s) [upload=4621]: Waiting for error group managing segments... Oct 27 19:56:59.801926071 (5h14m17.151233347s) [noise=true , address=81.171.14.70:4034 , piece_id=QQXBWVXWX24ETMCIFZ7LTONF3LVGL3RLDXXSIQGX62PLWPJSFM5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.811249835 (5h14m17.160557121s) [noise=true , address=81.171.14.70:4048 , piece_id=PR2HDQTP5VKTMF7HG6VNKKUJD66LC2HKQVAZUGEZKKZNKH2QCZFQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.858197924 (5h14m17.20750521s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.897902089 (5h14m17.247209375s) [noise=true , address=89.149.201.147:4023 , piece_id=LEEWZDLYWPSSUDRIPEQK4NKC4PFF6JVLEBPZMQJLWCG4YZHRFQ3A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.901773520 (5h14m17.251080806s) [noise=true , address=89.149.201.147:4010 , piece_id=TRY6HG262YJRDUVNLLHRYG554BVP4J5TWPGHJUVN5N55WHU3C4ZA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902238556 (5h14m17.251545842s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902438327 (5h14m17.251745603s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.917540715 (5h14m17.266847991s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.926674337 (5h14m17.275981623s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.328819374 (5h14m17.67812666s) [noise=true , address=81.171.14.70:4030 , piece_id=ZTVTBK2E7IGYOYZMK6TTHKMMBST52RCZVKM776PV3USLXONMH66Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.349296225 (5h14m17.698603501s) [noise=true , address=89.149.201.147:4011 , piece_id=KQM7L432OU7BYKWL2QHAIKRNPKPA7NOPSZJDOAAFQEGV66PZMGTA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.352514193 (5h14m17.701821479s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.380253194 (5h14m17.72956049s) [noise=true , address=185.167.204.205:4030 , piece_id=35Y5DRDZZHA7LEUZDJQ36RU4MB6DQN2AAIG7E45EIC6YM3MQ7HHQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.385432055 (5h14m17.734739341s) [noise=true , address=185.167.204.205:4023 , piece_id=ACMLGJMZ5H25VH7VULKAIBWIHIAEE2AV366UEZFOJDZRNQ4NGYLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.388082382 (5h14m17.737389678s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.391016018 (5h14m17.740323304s) [noise=true , address=185.167.204.206:4056 , piece_id=G5D6SI6CVQHAVSAPS6GJRVZIQ4YQ23PFH7KL26TVK4I4BBTW2C5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.396014777 (5h14m17.745322053s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.401559615 (5h14m17.750866901s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.414355180 (5h14m17.763662466s) [noise=true , address=185.167.204.206:4041 , piece_id=NUWDUCCNI44ANZLNYISQQKFSBK7WYTPAPGQ6JJ43IP44LNCLT4SA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.421539241 (5h14m17.770846537s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.423334288 (5h14m17.772641574s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.447102799 (5h14m17.796410075s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.473754258 (5h14m17.823061544s) [noise=true , address=185.167.204.206:4066 , piece_id=KKDV46RFAENAUYDYRPDCV64LWX5RB7JVRKB3HY7BXB4LU7DITSLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.481151965 (5h14m17.830459251s) [noise=true , address=185.167.204.205:4019 , piece_id=7LYDTS6D3HQSDMVBCMRLB5NZIRLP44VRSWCUT5EKSGSRPWP4RHIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.486154732 (5h14m17.835462018s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:57.813202781 (5h15m15.162510068s) [noise=true , address=185.167.204.206:4053 , piece_id=UFA7HABPN4LRP7RSCHBHNXT5AMEDFUF3DSYEGWBFSXOTOXRQXT7A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: rpc: dial tcp 185.167.204.206:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813463338 (5h15m15.162770614s) [noise=true , address=185.167.204.206:4047 , piece_id=Z2QX5FHJKEZVW4FXDV2NNUMXEBVPGEW4Z7C3GBVZRUTNFR4MA5IA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: write tcp 188.166.225.170:59506->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813491391 (5h15m15.162798677s) [noise=true , address=185.167.204.205:4013 , piece_id=KICB6KLFDM3KFOGJ52UVXO3G4BUPSSJF657ZIWUIVP56M2SLGMWA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: context deadline exceeded; context deadline exceeded Oct 27 19:57:57.813536527 (5h15m15.162843813s) [seg_pos={0 0}, upload=4621]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Oct 27 19:57:58.016780515 (5h15m15.366087802s) [upload=4621]: Done uploading segments... Oct 27 19:57:58.016879354 (5h15m15.36618663s) [upload=4621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 4.963811654s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 3.486804198s FAIL (2025-10-27 20:26:01 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.486831629s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.07482311s FAIL (2025-10-27 20:25:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== eg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577374904 (5h14m15.926682211s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577388009 (5h14m15.926695285s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577699713 (5h14m15.927006989s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577834229 (5h14m15.927141505s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578014502 (5h14m15.927321778s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578280880 (5h14m15.927588156s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578608904 (5h14m15.92791618s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578935426 (5h14m15.928242712s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579041648 (5h14m15.928348924s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579222723 (5h14m15.928529999s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579486625 (5h14m15.928793901s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579780545 (5h14m15.929087821s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.580102097 (5h14m15.929409403s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:59.402940250 (5h14m16.752247547s) [upload=4621]: Next returned nil segment Oct 27 19:56:59.402970929 (5h14m16.752278235s) [upload=4621]: Waiting for error group managing segments... Oct 27 19:56:59.801926071 (5h14m17.151233347s) [noise=true , address=81.171.14.70:4034 , piece_id=QQXBWVXWX24ETMCIFZ7LTONF3LVGL3RLDXXSIQGX62PLWPJSFM5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.811249835 (5h14m17.160557121s) [noise=true , address=81.171.14.70:4048 , piece_id=PR2HDQTP5VKTMF7HG6VNKKUJD66LC2HKQVAZUGEZKKZNKH2QCZFQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.858197924 (5h14m17.20750521s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.897902089 (5h14m17.247209375s) [noise=true , address=89.149.201.147:4023 , piece_id=LEEWZDLYWPSSUDRIPEQK4NKC4PFF6JVLEBPZMQJLWCG4YZHRFQ3A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.901773520 (5h14m17.251080806s) [noise=true , address=89.149.201.147:4010 , piece_id=TRY6HG262YJRDUVNLLHRYG554BVP4J5TWPGHJUVN5N55WHU3C4ZA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902238556 (5h14m17.251545842s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902438327 (5h14m17.251745603s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.917540715 (5h14m17.266847991s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.926674337 (5h14m17.275981623s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.328819374 (5h14m17.67812666s) [noise=true , address=81.171.14.70:4030 , piece_id=ZTVTBK2E7IGYOYZMK6TTHKMMBST52RCZVKM776PV3USLXONMH66Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.349296225 (5h14m17.698603501s) [noise=true , address=89.149.201.147:4011 , piece_id=KQM7L432OU7BYKWL2QHAIKRNPKPA7NOPSZJDOAAFQEGV66PZMGTA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.352514193 (5h14m17.701821479s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.380253194 (5h14m17.72956049s) [noise=true , address=185.167.204.205:4030 , piece_id=35Y5DRDZZHA7LEUZDJQ36RU4MB6DQN2AAIG7E45EIC6YM3MQ7HHQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.385432055 (5h14m17.734739341s) [noise=true , address=185.167.204.205:4023 , piece_id=ACMLGJMZ5H25VH7VULKAIBWIHIAEE2AV366UEZFOJDZRNQ4NGYLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.388082382 (5h14m17.737389678s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.391016018 (5h14m17.740323304s) [noise=true , address=185.167.204.206:4056 , piece_id=G5D6SI6CVQHAVSAPS6GJRVZIQ4YQ23PFH7KL26TVK4I4BBTW2C5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.396014777 (5h14m17.745322053s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.401559615 (5h14m17.750866901s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.414355180 (5h14m17.763662466s) [noise=true , address=185.167.204.206:4041 , piece_id=NUWDUCCNI44ANZLNYISQQKFSBK7WYTPAPGQ6JJ43IP44LNCLT4SA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.421539241 (5h14m17.770846537s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.423334288 (5h14m17.772641574s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.447102799 (5h14m17.796410075s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.473754258 (5h14m17.823061544s) [noise=true , address=185.167.204.206:4066 , piece_id=KKDV46RFAENAUYDYRPDCV64LWX5RB7JVRKB3HY7BXB4LU7DITSLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.481151965 (5h14m17.830459251s) [noise=true , address=185.167.204.205:4019 , piece_id=7LYDTS6D3HQSDMVBCMRLB5NZIRLP44VRSWCUT5EKSGSRPWP4RHIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.486154732 (5h14m17.835462018s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:57.813202781 (5h15m15.162510068s) [noise=true , address=185.167.204.206:4053 , piece_id=UFA7HABPN4LRP7RSCHBHNXT5AMEDFUF3DSYEGWBFSXOTOXRQXT7A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: rpc: dial tcp 185.167.204.206:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813463338 (5h15m15.162770614s) [noise=true , address=185.167.204.206:4047 , piece_id=Z2QX5FHJKEZVW4FXDV2NNUMXEBVPGEW4Z7C3GBVZRUTNFR4MA5IA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: write tcp 188.166.225.170:59506->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813491391 (5h15m15.162798677s) [noise=true , address=185.167.204.205:4013 , piece_id=KICB6KLFDM3KFOGJ52UVXO3G4BUPSSJF657ZIWUIVP56M2SLGMWA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: context deadline exceeded; context deadline exceeded Oct 27 19:57:57.813536527 (5h15m15.162843813s) [seg_pos={0 0}, upload=4621]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Oct 27 19:57:58.016780515 (5h15m15.366087802s) [upload=4621]: Done uploading segments... Oct 27 19:57:58.016879354 (5h15m15.36618663s) [upload=4621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.074855962s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 3.418452511s FAIL (2025-10-27 20:25:30 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.418477329s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.002063059s FAIL (2025-10-27 20:24:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== eg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577374904 (5h14m15.926682211s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577388009 (5h14m15.926695285s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577699713 (5h14m15.927006989s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577834229 (5h14m15.927141505s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578014502 (5h14m15.927321778s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578280880 (5h14m15.927588156s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578608904 (5h14m15.92791618s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578935426 (5h14m15.928242712s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579041648 (5h14m15.928348924s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579222723 (5h14m15.928529999s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579486625 (5h14m15.928793901s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579780545 (5h14m15.929087821s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.580102097 (5h14m15.929409403s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:59.402940250 (5h14m16.752247547s) [upload=4621]: Next returned nil segment Oct 27 19:56:59.402970929 (5h14m16.752278235s) [upload=4621]: Waiting for error group managing segments... Oct 27 19:56:59.801926071 (5h14m17.151233347s) [noise=true , address=81.171.14.70:4034 , piece_id=QQXBWVXWX24ETMCIFZ7LTONF3LVGL3RLDXXSIQGX62PLWPJSFM5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.811249835 (5h14m17.160557121s) [noise=true , address=81.171.14.70:4048 , piece_id=PR2HDQTP5VKTMF7HG6VNKKUJD66LC2HKQVAZUGEZKKZNKH2QCZFQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.858197924 (5h14m17.20750521s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.897902089 (5h14m17.247209375s) [noise=true , address=89.149.201.147:4023 , piece_id=LEEWZDLYWPSSUDRIPEQK4NKC4PFF6JVLEBPZMQJLWCG4YZHRFQ3A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.901773520 (5h14m17.251080806s) [noise=true , address=89.149.201.147:4010 , piece_id=TRY6HG262YJRDUVNLLHRYG554BVP4J5TWPGHJUVN5N55WHU3C4ZA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902238556 (5h14m17.251545842s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902438327 (5h14m17.251745603s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.917540715 (5h14m17.266847991s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.926674337 (5h14m17.275981623s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.328819374 (5h14m17.67812666s) [noise=true , address=81.171.14.70:4030 , piece_id=ZTVTBK2E7IGYOYZMK6TTHKMMBST52RCZVKM776PV3USLXONMH66Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.349296225 (5h14m17.698603501s) [noise=true , address=89.149.201.147:4011 , piece_id=KQM7L432OU7BYKWL2QHAIKRNPKPA7NOPSZJDOAAFQEGV66PZMGTA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.352514193 (5h14m17.701821479s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.380253194 (5h14m17.72956049s) [noise=true , address=185.167.204.205:4030 , piece_id=35Y5DRDZZHA7LEUZDJQ36RU4MB6DQN2AAIG7E45EIC6YM3MQ7HHQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.385432055 (5h14m17.734739341s) [noise=true , address=185.167.204.205:4023 , piece_id=ACMLGJMZ5H25VH7VULKAIBWIHIAEE2AV366UEZFOJDZRNQ4NGYLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.388082382 (5h14m17.737389678s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.391016018 (5h14m17.740323304s) [noise=true , address=185.167.204.206:4056 , piece_id=G5D6SI6CVQHAVSAPS6GJRVZIQ4YQ23PFH7KL26TVK4I4BBTW2C5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.396014777 (5h14m17.745322053s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.401559615 (5h14m17.750866901s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.414355180 (5h14m17.763662466s) [noise=true , address=185.167.204.206:4041 , piece_id=NUWDUCCNI44ANZLNYISQQKFSBK7WYTPAPGQ6JJ43IP44LNCLT4SA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.421539241 (5h14m17.770846537s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.423334288 (5h14m17.772641574s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.447102799 (5h14m17.796410075s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.473754258 (5h14m17.823061544s) [noise=true , address=185.167.204.206:4066 , piece_id=KKDV46RFAENAUYDYRPDCV64LWX5RB7JVRKB3HY7BXB4LU7DITSLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.481151965 (5h14m17.830459251s) [noise=true , address=185.167.204.205:4019 , piece_id=7LYDTS6D3HQSDMVBCMRLB5NZIRLP44VRSWCUT5EKSGSRPWP4RHIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.486154732 (5h14m17.835462018s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:57.813202781 (5h15m15.162510068s) [noise=true , address=185.167.204.206:4053 , piece_id=UFA7HABPN4LRP7RSCHBHNXT5AMEDFUF3DSYEGWBFSXOTOXRQXT7A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: rpc: dial tcp 185.167.204.206:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813463338 (5h15m15.162770614s) [noise=true , address=185.167.204.206:4047 , piece_id=Z2QX5FHJKEZVW4FXDV2NNUMXEBVPGEW4Z7C3GBVZRUTNFR4MA5IA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: write tcp 188.166.225.170:59506->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813491391 (5h15m15.162798677s) [noise=true , address=185.167.204.205:4013 , piece_id=KICB6KLFDM3KFOGJ52UVXO3G4BUPSSJF657ZIWUIVP56M2SLGMWA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: context deadline exceeded; context deadline exceeded Oct 27 19:57:57.813536527 (5h15m15.162843813s) [seg_pos={0 0}, upload=4621]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Oct 27 19:57:58.016780515 (5h15m15.366087802s) [upload=4621]: Done uploading segments... Oct 27 19:57:58.016879354 (5h15m15.36618663s) [upload=4621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.002094148s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 3.53626518s FAIL (2025-10-27 20:24:16 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.53629687s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.005107676s FAIL (2025-10-27 20:24:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== eg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577374904 (5h14m15.926682211s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577388009 (5h14m15.926695285s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577699713 (5h14m15.927006989s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577834229 (5h14m15.927141505s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578014502 (5h14m15.927321778s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578280880 (5h14m15.927588156s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578608904 (5h14m15.92791618s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578935426 (5h14m15.928242712s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579041648 (5h14m15.928348924s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579222723 (5h14m15.928529999s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579486625 (5h14m15.928793901s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579780545 (5h14m15.929087821s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.580102097 (5h14m15.929409403s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:59.402940250 (5h14m16.752247547s) [upload=4621]: Next returned nil segment Oct 27 19:56:59.402970929 (5h14m16.752278235s) [upload=4621]: Waiting for error group managing segments... Oct 27 19:56:59.801926071 (5h14m17.151233347s) [noise=true , address=81.171.14.70:4034 , piece_id=QQXBWVXWX24ETMCIFZ7LTONF3LVGL3RLDXXSIQGX62PLWPJSFM5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.811249835 (5h14m17.160557121s) [noise=true , address=81.171.14.70:4048 , piece_id=PR2HDQTP5VKTMF7HG6VNKKUJD66LC2HKQVAZUGEZKKZNKH2QCZFQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.858197924 (5h14m17.20750521s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.897902089 (5h14m17.247209375s) [noise=true , address=89.149.201.147:4023 , piece_id=LEEWZDLYWPSSUDRIPEQK4NKC4PFF6JVLEBPZMQJLWCG4YZHRFQ3A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.901773520 (5h14m17.251080806s) [noise=true , address=89.149.201.147:4010 , piece_id=TRY6HG262YJRDUVNLLHRYG554BVP4J5TWPGHJUVN5N55WHU3C4ZA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902238556 (5h14m17.251545842s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902438327 (5h14m17.251745603s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.917540715 (5h14m17.266847991s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.926674337 (5h14m17.275981623s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.328819374 (5h14m17.67812666s) [noise=true , address=81.171.14.70:4030 , piece_id=ZTVTBK2E7IGYOYZMK6TTHKMMBST52RCZVKM776PV3USLXONMH66Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.349296225 (5h14m17.698603501s) [noise=true , address=89.149.201.147:4011 , piece_id=KQM7L432OU7BYKWL2QHAIKRNPKPA7NOPSZJDOAAFQEGV66PZMGTA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.352514193 (5h14m17.701821479s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.380253194 (5h14m17.72956049s) [noise=true , address=185.167.204.205:4030 , piece_id=35Y5DRDZZHA7LEUZDJQ36RU4MB6DQN2AAIG7E45EIC6YM3MQ7HHQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.385432055 (5h14m17.734739341s) [noise=true , address=185.167.204.205:4023 , piece_id=ACMLGJMZ5H25VH7VULKAIBWIHIAEE2AV366UEZFOJDZRNQ4NGYLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.388082382 (5h14m17.737389678s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.391016018 (5h14m17.740323304s) [noise=true , address=185.167.204.206:4056 , piece_id=G5D6SI6CVQHAVSAPS6GJRVZIQ4YQ23PFH7KL26TVK4I4BBTW2C5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.396014777 (5h14m17.745322053s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.401559615 (5h14m17.750866901s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.414355180 (5h14m17.763662466s) [noise=true , address=185.167.204.206:4041 , piece_id=NUWDUCCNI44ANZLNYISQQKFSBK7WYTPAPGQ6JJ43IP44LNCLT4SA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.421539241 (5h14m17.770846537s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.423334288 (5h14m17.772641574s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.447102799 (5h14m17.796410075s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.473754258 (5h14m17.823061544s) [noise=true , address=185.167.204.206:4066 , piece_id=KKDV46RFAENAUYDYRPDCV64LWX5RB7JVRKB3HY7BXB4LU7DITSLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.481151965 (5h14m17.830459251s) [noise=true , address=185.167.204.205:4019 , piece_id=7LYDTS6D3HQSDMVBCMRLB5NZIRLP44VRSWCUT5EKSGSRPWP4RHIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.486154732 (5h14m17.835462018s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:57.813202781 (5h15m15.162510068s) [noise=true , address=185.167.204.206:4053 , piece_id=UFA7HABPN4LRP7RSCHBHNXT5AMEDFUF3DSYEGWBFSXOTOXRQXT7A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: rpc: dial tcp 185.167.204.206:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813463338 (5h15m15.162770614s) [noise=true , address=185.167.204.206:4047 , piece_id=Z2QX5FHJKEZVW4FXDV2NNUMXEBVPGEW4Z7C3GBVZRUTNFR4MA5IA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: write tcp 188.166.225.170:59506->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813491391 (5h15m15.162798677s) [noise=true , address=185.167.204.205:4013 , piece_id=KICB6KLFDM3KFOGJ52UVXO3G4BUPSSJF657ZIWUIVP56M2SLGMWA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: context deadline exceeded; context deadline exceeded Oct 27 19:57:57.813536527 (5h15m15.162843813s) [seg_pos={0 0}, upload=4621]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Oct 27 19:57:58.016780515 (5h15m15.366087802s) [upload=4621]: Done uploading segments... Oct 27 19:57:58.016879354 (5h15m15.36618663s) [upload=4621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.005144678s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 3.355724308s FAIL (2025-10-27 20:23:58 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.355754975s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.001796284s FAIL (2025-10-27 20:23:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== eg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577374904 (5h14m15.926682211s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577388009 (5h14m15.926695285s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577699713 (5h14m15.927006989s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577834229 (5h14m15.927141505s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578014502 (5h14m15.927321778s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578280880 (5h14m15.927588156s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578608904 (5h14m15.92791618s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578935426 (5h14m15.928242712s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579041648 (5h14m15.928348924s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579222723 (5h14m15.928529999s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579486625 (5h14m15.928793901s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579780545 (5h14m15.929087821s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.580102097 (5h14m15.929409403s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:59.402940250 (5h14m16.752247547s) [upload=4621]: Next returned nil segment Oct 27 19:56:59.402970929 (5h14m16.752278235s) [upload=4621]: Waiting for error group managing segments... Oct 27 19:56:59.801926071 (5h14m17.151233347s) [noise=true , address=81.171.14.70:4034 , piece_id=QQXBWVXWX24ETMCIFZ7LTONF3LVGL3RLDXXSIQGX62PLWPJSFM5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.811249835 (5h14m17.160557121s) [noise=true , address=81.171.14.70:4048 , piece_id=PR2HDQTP5VKTMF7HG6VNKKUJD66LC2HKQVAZUGEZKKZNKH2QCZFQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.858197924 (5h14m17.20750521s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.897902089 (5h14m17.247209375s) [noise=true , address=89.149.201.147:4023 , piece_id=LEEWZDLYWPSSUDRIPEQK4NKC4PFF6JVLEBPZMQJLWCG4YZHRFQ3A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.901773520 (5h14m17.251080806s) [noise=true , address=89.149.201.147:4010 , piece_id=TRY6HG262YJRDUVNLLHRYG554BVP4J5TWPGHJUVN5N55WHU3C4ZA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902238556 (5h14m17.251545842s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902438327 (5h14m17.251745603s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.917540715 (5h14m17.266847991s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.926674337 (5h14m17.275981623s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.328819374 (5h14m17.67812666s) [noise=true , address=81.171.14.70:4030 , piece_id=ZTVTBK2E7IGYOYZMK6TTHKMMBST52RCZVKM776PV3USLXONMH66Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.349296225 (5h14m17.698603501s) [noise=true , address=89.149.201.147:4011 , piece_id=KQM7L432OU7BYKWL2QHAIKRNPKPA7NOPSZJDOAAFQEGV66PZMGTA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.352514193 (5h14m17.701821479s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.380253194 (5h14m17.72956049s) [noise=true , address=185.167.204.205:4030 , piece_id=35Y5DRDZZHA7LEUZDJQ36RU4MB6DQN2AAIG7E45EIC6YM3MQ7HHQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.385432055 (5h14m17.734739341s) [noise=true , address=185.167.204.205:4023 , piece_id=ACMLGJMZ5H25VH7VULKAIBWIHIAEE2AV366UEZFOJDZRNQ4NGYLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.388082382 (5h14m17.737389678s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.391016018 (5h14m17.740323304s) [noise=true , address=185.167.204.206:4056 , piece_id=G5D6SI6CVQHAVSAPS6GJRVZIQ4YQ23PFH7KL26TVK4I4BBTW2C5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.396014777 (5h14m17.745322053s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.401559615 (5h14m17.750866901s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.414355180 (5h14m17.763662466s) [noise=true , address=185.167.204.206:4041 , piece_id=NUWDUCCNI44ANZLNYISQQKFSBK7WYTPAPGQ6JJ43IP44LNCLT4SA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.421539241 (5h14m17.770846537s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.423334288 (5h14m17.772641574s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.447102799 (5h14m17.796410075s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.473754258 (5h14m17.823061544s) [noise=true , address=185.167.204.206:4066 , piece_id=KKDV46RFAENAUYDYRPDCV64LWX5RB7JVRKB3HY7BXB4LU7DITSLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.481151965 (5h14m17.830459251s) [noise=true , address=185.167.204.205:4019 , piece_id=7LYDTS6D3HQSDMVBCMRLB5NZIRLP44VRSWCUT5EKSGSRPWP4RHIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.486154732 (5h14m17.835462018s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:57.813202781 (5h15m15.162510068s) [noise=true , address=185.167.204.206:4053 , piece_id=UFA7HABPN4LRP7RSCHBHNXT5AMEDFUF3DSYEGWBFSXOTOXRQXT7A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: rpc: dial tcp 185.167.204.206:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813463338 (5h15m15.162770614s) [noise=true , address=185.167.204.206:4047 , piece_id=Z2QX5FHJKEZVW4FXDV2NNUMXEBVPGEW4Z7C3GBVZRUTNFR4MA5IA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: write tcp 188.166.225.170:59506->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813491391 (5h15m15.162798677s) [noise=true , address=185.167.204.205:4013 , piece_id=KICB6KLFDM3KFOGJ52UVXO3G4BUPSSJF657ZIWUIVP56M2SLGMWA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: context deadline exceeded; context deadline exceeded Oct 27 19:57:57.813536527 (5h15m15.162843813s) [seg_pos={0 0}, upload=4621]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Oct 27 19:57:58.016780515 (5h15m15.366087802s) [upload=4621]: Done uploading segments... Oct 27 19:57:58.016879354 (5h15m15.36618663s) [upload=4621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.001818377s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 4.985220578s FAIL (2025-10-27 20:22:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== eg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577374904 (5h14m15.926682211s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577388009 (5h14m15.926695285s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577699713 (5h14m15.927006989s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577834229 (5h14m15.927141505s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578014502 (5h14m15.927321778s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578280880 (5h14m15.927588156s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578608904 (5h14m15.92791618s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578935426 (5h14m15.928242712s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579041648 (5h14m15.928348924s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579222723 (5h14m15.928529999s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579486625 (5h14m15.928793901s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579780545 (5h14m15.929087821s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.580102097 (5h14m15.929409403s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:59.402940250 (5h14m16.752247547s) [upload=4621]: Next returned nil segment Oct 27 19:56:59.402970929 (5h14m16.752278235s) [upload=4621]: Waiting for error group managing segments... Oct 27 19:56:59.801926071 (5h14m17.151233347s) [noise=true , address=81.171.14.70:4034 , piece_id=QQXBWVXWX24ETMCIFZ7LTONF3LVGL3RLDXXSIQGX62PLWPJSFM5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.811249835 (5h14m17.160557121s) [noise=true , address=81.171.14.70:4048 , piece_id=PR2HDQTP5VKTMF7HG6VNKKUJD66LC2HKQVAZUGEZKKZNKH2QCZFQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.858197924 (5h14m17.20750521s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.897902089 (5h14m17.247209375s) [noise=true , address=89.149.201.147:4023 , piece_id=LEEWZDLYWPSSUDRIPEQK4NKC4PFF6JVLEBPZMQJLWCG4YZHRFQ3A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.901773520 (5h14m17.251080806s) [noise=true , address=89.149.201.147:4010 , piece_id=TRY6HG262YJRDUVNLLHRYG554BVP4J5TWPGHJUVN5N55WHU3C4ZA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902238556 (5h14m17.251545842s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902438327 (5h14m17.251745603s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.917540715 (5h14m17.266847991s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.926674337 (5h14m17.275981623s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.328819374 (5h14m17.67812666s) [noise=true , address=81.171.14.70:4030 , piece_id=ZTVTBK2E7IGYOYZMK6TTHKMMBST52RCZVKM776PV3USLXONMH66Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.349296225 (5h14m17.698603501s) [noise=true , address=89.149.201.147:4011 , piece_id=KQM7L432OU7BYKWL2QHAIKRNPKPA7NOPSZJDOAAFQEGV66PZMGTA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.352514193 (5h14m17.701821479s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.380253194 (5h14m17.72956049s) [noise=true , address=185.167.204.205:4030 , piece_id=35Y5DRDZZHA7LEUZDJQ36RU4MB6DQN2AAIG7E45EIC6YM3MQ7HHQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.385432055 (5h14m17.734739341s) [noise=true , address=185.167.204.205:4023 , piece_id=ACMLGJMZ5H25VH7VULKAIBWIHIAEE2AV366UEZFOJDZRNQ4NGYLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.388082382 (5h14m17.737389678s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.391016018 (5h14m17.740323304s) [noise=true , address=185.167.204.206:4056 , piece_id=G5D6SI6CVQHAVSAPS6GJRVZIQ4YQ23PFH7KL26TVK4I4BBTW2C5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.396014777 (5h14m17.745322053s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.401559615 (5h14m17.750866901s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.414355180 (5h14m17.763662466s) [noise=true , address=185.167.204.206:4041 , piece_id=NUWDUCCNI44ANZLNYISQQKFSBK7WYTPAPGQ6JJ43IP44LNCLT4SA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.421539241 (5h14m17.770846537s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.423334288 (5h14m17.772641574s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.447102799 (5h14m17.796410075s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.473754258 (5h14m17.823061544s) [noise=true , address=185.167.204.206:4066 , piece_id=KKDV46RFAENAUYDYRPDCV64LWX5RB7JVRKB3HY7BXB4LU7DITSLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.481151965 (5h14m17.830459251s) [noise=true , address=185.167.204.205:4019 , piece_id=7LYDTS6D3HQSDMVBCMRLB5NZIRLP44VRSWCUT5EKSGSRPWP4RHIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.486154732 (5h14m17.835462018s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:57.813202781 (5h15m15.162510068s) [noise=true , address=185.167.204.206:4053 , piece_id=UFA7HABPN4LRP7RSCHBHNXT5AMEDFUF3DSYEGWBFSXOTOXRQXT7A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: rpc: dial tcp 185.167.204.206:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813463338 (5h15m15.162770614s) [noise=true , address=185.167.204.206:4047 , piece_id=Z2QX5FHJKEZVW4FXDV2NNUMXEBVPGEW4Z7C3GBVZRUTNFR4MA5IA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: write tcp 188.166.225.170:59506->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813491391 (5h15m15.162798677s) [noise=true , address=185.167.204.205:4013 , piece_id=KICB6KLFDM3KFOGJ52UVXO3G4BUPSSJF657ZIWUIVP56M2SLGMWA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: context deadline exceeded; context deadline exceeded Oct 27 19:57:57.813536527 (5h15m15.162843813s) [seg_pos={0 0}, upload=4621]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Oct 27 19:57:58.016780515 (5h15m15.366087802s) [upload=4621]: Done uploading segments... Oct 27 19:57:58.016879354 (5h15m15.36618663s) [upload=4621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 4.985249964s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 3.404615121s FAIL (2025-10-27 20:22:47 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.404643374s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 3.725608455s FAIL (2025-10-27 20:22:31 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.72563734s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 4.95487694s FAIL (2025-10-27 20:22:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== eg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577374904 (5h14m15.926682211s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577388009 (5h14m15.926695285s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577699713 (5h14m15.927006989s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577834229 (5h14m15.927141505s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578014502 (5h14m15.927321778s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578280880 (5h14m15.927588156s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578608904 (5h14m15.92791618s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578935426 (5h14m15.928242712s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579041648 (5h14m15.928348924s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579222723 (5h14m15.928529999s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579486625 (5h14m15.928793901s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579780545 (5h14m15.929087821s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.580102097 (5h14m15.929409403s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:59.402940250 (5h14m16.752247547s) [upload=4621]: Next returned nil segment Oct 27 19:56:59.402970929 (5h14m16.752278235s) [upload=4621]: Waiting for error group managing segments... Oct 27 19:56:59.801926071 (5h14m17.151233347s) [noise=true , address=81.171.14.70:4034 , piece_id=QQXBWVXWX24ETMCIFZ7LTONF3LVGL3RLDXXSIQGX62PLWPJSFM5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.811249835 (5h14m17.160557121s) [noise=true , address=81.171.14.70:4048 , piece_id=PR2HDQTP5VKTMF7HG6VNKKUJD66LC2HKQVAZUGEZKKZNKH2QCZFQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.858197924 (5h14m17.20750521s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.897902089 (5h14m17.247209375s) [noise=true , address=89.149.201.147:4023 , piece_id=LEEWZDLYWPSSUDRIPEQK4NKC4PFF6JVLEBPZMQJLWCG4YZHRFQ3A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.901773520 (5h14m17.251080806s) [noise=true , address=89.149.201.147:4010 , piece_id=TRY6HG262YJRDUVNLLHRYG554BVP4J5TWPGHJUVN5N55WHU3C4ZA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902238556 (5h14m17.251545842s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902438327 (5h14m17.251745603s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.917540715 (5h14m17.266847991s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.926674337 (5h14m17.275981623s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.328819374 (5h14m17.67812666s) [noise=true , address=81.171.14.70:4030 , piece_id=ZTVTBK2E7IGYOYZMK6TTHKMMBST52RCZVKM776PV3USLXONMH66Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.349296225 (5h14m17.698603501s) [noise=true , address=89.149.201.147:4011 , piece_id=KQM7L432OU7BYKWL2QHAIKRNPKPA7NOPSZJDOAAFQEGV66PZMGTA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.352514193 (5h14m17.701821479s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.380253194 (5h14m17.72956049s) [noise=true , address=185.167.204.205:4030 , piece_id=35Y5DRDZZHA7LEUZDJQ36RU4MB6DQN2AAIG7E45EIC6YM3MQ7HHQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.385432055 (5h14m17.734739341s) [noise=true , address=185.167.204.205:4023 , piece_id=ACMLGJMZ5H25VH7VULKAIBWIHIAEE2AV366UEZFOJDZRNQ4NGYLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.388082382 (5h14m17.737389678s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.391016018 (5h14m17.740323304s) [noise=true , address=185.167.204.206:4056 , piece_id=G5D6SI6CVQHAVSAPS6GJRVZIQ4YQ23PFH7KL26TVK4I4BBTW2C5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.396014777 (5h14m17.745322053s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.401559615 (5h14m17.750866901s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.414355180 (5h14m17.763662466s) [noise=true , address=185.167.204.206:4041 , piece_id=NUWDUCCNI44ANZLNYISQQKFSBK7WYTPAPGQ6JJ43IP44LNCLT4SA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.421539241 (5h14m17.770846537s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.423334288 (5h14m17.772641574s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.447102799 (5h14m17.796410075s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.473754258 (5h14m17.823061544s) [noise=true , address=185.167.204.206:4066 , piece_id=KKDV46RFAENAUYDYRPDCV64LWX5RB7JVRKB3HY7BXB4LU7DITSLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.481151965 (5h14m17.830459251s) [noise=true , address=185.167.204.205:4019 , piece_id=7LYDTS6D3HQSDMVBCMRLB5NZIRLP44VRSWCUT5EKSGSRPWP4RHIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.486154732 (5h14m17.835462018s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:57.813202781 (5h15m15.162510068s) [noise=true , address=185.167.204.206:4053 , piece_id=UFA7HABPN4LRP7RSCHBHNXT5AMEDFUF3DSYEGWBFSXOTOXRQXT7A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: rpc: dial tcp 185.167.204.206:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813463338 (5h15m15.162770614s) [noise=true , address=185.167.204.206:4047 , piece_id=Z2QX5FHJKEZVW4FXDV2NNUMXEBVPGEW4Z7C3GBVZRUTNFR4MA5IA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: write tcp 188.166.225.170:59506->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813491391 (5h15m15.162798677s) [noise=true , address=185.167.204.205:4013 , piece_id=KICB6KLFDM3KFOGJ52UVXO3G4BUPSSJF657ZIWUIVP56M2SLGMWA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: context deadline exceeded; context deadline exceeded Oct 27 19:57:57.813536527 (5h15m15.162843813s) [seg_pos={0 0}, upload=4621]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Oct 27 19:57:58.016780515 (5h15m15.366087802s) [upload=4621]: Done uploading segments... Oct 27 19:57:58.016879354 (5h15m15.36618663s) [upload=4621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 4.954904643s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 3.490657479s FAIL (2025-10-27 20:21:50 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.490686464s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.060135089s FAIL (2025-10-27 20:21:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== eg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577374904 (5h14m15.926682211s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577388009 (5h14m15.926695285s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577699713 (5h14m15.927006989s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577834229 (5h14m15.927141505s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578014502 (5h14m15.927321778s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578280880 (5h14m15.927588156s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578608904 (5h14m15.92791618s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578935426 (5h14m15.928242712s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579041648 (5h14m15.928348924s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579222723 (5h14m15.928529999s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579486625 (5h14m15.928793901s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579780545 (5h14m15.929087821s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.580102097 (5h14m15.929409403s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:59.402940250 (5h14m16.752247547s) [upload=4621]: Next returned nil segment Oct 27 19:56:59.402970929 (5h14m16.752278235s) [upload=4621]: Waiting for error group managing segments... Oct 27 19:56:59.801926071 (5h14m17.151233347s) [noise=true , address=81.171.14.70:4034 , piece_id=QQXBWVXWX24ETMCIFZ7LTONF3LVGL3RLDXXSIQGX62PLWPJSFM5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.811249835 (5h14m17.160557121s) [noise=true , address=81.171.14.70:4048 , piece_id=PR2HDQTP5VKTMF7HG6VNKKUJD66LC2HKQVAZUGEZKKZNKH2QCZFQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.858197924 (5h14m17.20750521s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.897902089 (5h14m17.247209375s) [noise=true , address=89.149.201.147:4023 , piece_id=LEEWZDLYWPSSUDRIPEQK4NKC4PFF6JVLEBPZMQJLWCG4YZHRFQ3A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.901773520 (5h14m17.251080806s) [noise=true , address=89.149.201.147:4010 , piece_id=TRY6HG262YJRDUVNLLHRYG554BVP4J5TWPGHJUVN5N55WHU3C4ZA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902238556 (5h14m17.251545842s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902438327 (5h14m17.251745603s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.917540715 (5h14m17.266847991s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.926674337 (5h14m17.275981623s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.328819374 (5h14m17.67812666s) [noise=true , address=81.171.14.70:4030 , piece_id=ZTVTBK2E7IGYOYZMK6TTHKMMBST52RCZVKM776PV3USLXONMH66Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.349296225 (5h14m17.698603501s) [noise=true , address=89.149.201.147:4011 , piece_id=KQM7L432OU7BYKWL2QHAIKRNPKPA7NOPSZJDOAAFQEGV66PZMGTA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.352514193 (5h14m17.701821479s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.380253194 (5h14m17.72956049s) [noise=true , address=185.167.204.205:4030 , piece_id=35Y5DRDZZHA7LEUZDJQ36RU4MB6DQN2AAIG7E45EIC6YM3MQ7HHQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.385432055 (5h14m17.734739341s) [noise=true , address=185.167.204.205:4023 , piece_id=ACMLGJMZ5H25VH7VULKAIBWIHIAEE2AV366UEZFOJDZRNQ4NGYLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.388082382 (5h14m17.737389678s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.391016018 (5h14m17.740323304s) [noise=true , address=185.167.204.206:4056 , piece_id=G5D6SI6CVQHAVSAPS6GJRVZIQ4YQ23PFH7KL26TVK4I4BBTW2C5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.396014777 (5h14m17.745322053s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.401559615 (5h14m17.750866901s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.414355180 (5h14m17.763662466s) [noise=true , address=185.167.204.206:4041 , piece_id=NUWDUCCNI44ANZLNYISQQKFSBK7WYTPAPGQ6JJ43IP44LNCLT4SA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.421539241 (5h14m17.770846537s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.423334288 (5h14m17.772641574s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.447102799 (5h14m17.796410075s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.473754258 (5h14m17.823061544s) [noise=true , address=185.167.204.206:4066 , piece_id=KKDV46RFAENAUYDYRPDCV64LWX5RB7JVRKB3HY7BXB4LU7DITSLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.481151965 (5h14m17.830459251s) [noise=true , address=185.167.204.205:4019 , piece_id=7LYDTS6D3HQSDMVBCMRLB5NZIRLP44VRSWCUT5EKSGSRPWP4RHIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.486154732 (5h14m17.835462018s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:57.813202781 (5h15m15.162510068s) [noise=true , address=185.167.204.206:4053 , piece_id=UFA7HABPN4LRP7RSCHBHNXT5AMEDFUF3DSYEGWBFSXOTOXRQXT7A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: rpc: dial tcp 185.167.204.206:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813463338 (5h15m15.162770614s) [noise=true , address=185.167.204.206:4047 , piece_id=Z2QX5FHJKEZVW4FXDV2NNUMXEBVPGEW4Z7C3GBVZRUTNFR4MA5IA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: write tcp 188.166.225.170:59506->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813491391 (5h15m15.162798677s) [noise=true , address=185.167.204.205:4013 , piece_id=KICB6KLFDM3KFOGJ52UVXO3G4BUPSSJF657ZIWUIVP56M2SLGMWA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: context deadline exceeded; context deadline exceeded Oct 27 19:57:57.813536527 (5h15m15.162843813s) [seg_pos={0 0}, upload=4621]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Oct 27 19:57:58.016780515 (5h15m15.366087802s) [upload=4621]: Done uploading segments... Oct 27 19:57:58.016879354 (5h15m15.36618663s) [upload=4621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.060169195s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 3.385983571s FAIL (2025-10-27 20:21:08 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.386129509s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 4.983570983s FAIL (2025-10-27 20:20:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== eg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577374904 (5h14m15.926682211s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577388009 (5h14m15.926695285s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577699713 (5h14m15.927006989s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577834229 (5h14m15.927141505s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578014502 (5h14m15.927321778s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578280880 (5h14m15.927588156s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578608904 (5h14m15.92791618s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578935426 (5h14m15.928242712s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579041648 (5h14m15.928348924s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579222723 (5h14m15.928529999s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579486625 (5h14m15.928793901s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579780545 (5h14m15.929087821s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.580102097 (5h14m15.929409403s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:59.402940250 (5h14m16.752247547s) [upload=4621]: Next returned nil segment Oct 27 19:56:59.402970929 (5h14m16.752278235s) [upload=4621]: Waiting for error group managing segments... Oct 27 19:56:59.801926071 (5h14m17.151233347s) [noise=true , address=81.171.14.70:4034 , piece_id=QQXBWVXWX24ETMCIFZ7LTONF3LVGL3RLDXXSIQGX62PLWPJSFM5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.811249835 (5h14m17.160557121s) [noise=true , address=81.171.14.70:4048 , piece_id=PR2HDQTP5VKTMF7HG6VNKKUJD66LC2HKQVAZUGEZKKZNKH2QCZFQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.858197924 (5h14m17.20750521s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.897902089 (5h14m17.247209375s) [noise=true , address=89.149.201.147:4023 , piece_id=LEEWZDLYWPSSUDRIPEQK4NKC4PFF6JVLEBPZMQJLWCG4YZHRFQ3A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.901773520 (5h14m17.251080806s) [noise=true , address=89.149.201.147:4010 , piece_id=TRY6HG262YJRDUVNLLHRYG554BVP4J5TWPGHJUVN5N55WHU3C4ZA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902238556 (5h14m17.251545842s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902438327 (5h14m17.251745603s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.917540715 (5h14m17.266847991s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.926674337 (5h14m17.275981623s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.328819374 (5h14m17.67812666s) [noise=true , address=81.171.14.70:4030 , piece_id=ZTVTBK2E7IGYOYZMK6TTHKMMBST52RCZVKM776PV3USLXONMH66Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.349296225 (5h14m17.698603501s) [noise=true , address=89.149.201.147:4011 , piece_id=KQM7L432OU7BYKWL2QHAIKRNPKPA7NOPSZJDOAAFQEGV66PZMGTA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.352514193 (5h14m17.701821479s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.380253194 (5h14m17.72956049s) [noise=true , address=185.167.204.205:4030 , piece_id=35Y5DRDZZHA7LEUZDJQ36RU4MB6DQN2AAIG7E45EIC6YM3MQ7HHQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.385432055 (5h14m17.734739341s) [noise=true , address=185.167.204.205:4023 , piece_id=ACMLGJMZ5H25VH7VULKAIBWIHIAEE2AV366UEZFOJDZRNQ4NGYLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.388082382 (5h14m17.737389678s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.391016018 (5h14m17.740323304s) [noise=true , address=185.167.204.206:4056 , piece_id=G5D6SI6CVQHAVSAPS6GJRVZIQ4YQ23PFH7KL26TVK4I4BBTW2C5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.396014777 (5h14m17.745322053s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.401559615 (5h14m17.750866901s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.414355180 (5h14m17.763662466s) [noise=true , address=185.167.204.206:4041 , piece_id=NUWDUCCNI44ANZLNYISQQKFSBK7WYTPAPGQ6JJ43IP44LNCLT4SA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.421539241 (5h14m17.770846537s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.423334288 (5h14m17.772641574s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.447102799 (5h14m17.796410075s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.473754258 (5h14m17.823061544s) [noise=true , address=185.167.204.206:4066 , piece_id=KKDV46RFAENAUYDYRPDCV64LWX5RB7JVRKB3HY7BXB4LU7DITSLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.481151965 (5h14m17.830459251s) [noise=true , address=185.167.204.205:4019 , piece_id=7LYDTS6D3HQSDMVBCMRLB5NZIRLP44VRSWCUT5EKSGSRPWP4RHIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.486154732 (5h14m17.835462018s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:57.813202781 (5h15m15.162510068s) [noise=true , address=185.167.204.206:4053 , piece_id=UFA7HABPN4LRP7RSCHBHNXT5AMEDFUF3DSYEGWBFSXOTOXRQXT7A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: rpc: dial tcp 185.167.204.206:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813463338 (5h15m15.162770614s) [noise=true , address=185.167.204.206:4047 , piece_id=Z2QX5FHJKEZVW4FXDV2NNUMXEBVPGEW4Z7C3GBVZRUTNFR4MA5IA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: write tcp 188.166.225.170:59506->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813491391 (5h15m15.162798677s) [noise=true , address=185.167.204.205:4013 , piece_id=KICB6KLFDM3KFOGJ52UVXO3G4BUPSSJF657ZIWUIVP56M2SLGMWA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: context deadline exceeded; context deadline exceeded Oct 27 19:57:57.813536527 (5h15m15.162843813s) [seg_pos={0 0}, upload=4621]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Oct 27 19:57:58.016780515 (5h15m15.366087802s) [upload=4621]: Done uploading segments... Oct 27 19:57:58.016879354 (5h15m15.36618663s) [upload=4621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 4.983608234s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 3.433233422s FAIL (2025-10-27 20:20:15 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.433259662s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 3.380371355s FAIL (2025-10-27 20:19:27 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.3803956s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.021580826s FAIL (2025-10-27 20:19:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== eg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577374904 (5h14m15.926682211s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577388009 (5h14m15.926695285s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577699713 (5h14m15.927006989s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577834229 (5h14m15.927141505s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578014502 (5h14m15.927321778s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578280880 (5h14m15.927588156s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578608904 (5h14m15.92791618s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578935426 (5h14m15.928242712s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579041648 (5h14m15.928348924s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579222723 (5h14m15.928529999s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579486625 (5h14m15.928793901s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579780545 (5h14m15.929087821s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.580102097 (5h14m15.929409403s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:59.402940250 (5h14m16.752247547s) [upload=4621]: Next returned nil segment Oct 27 19:56:59.402970929 (5h14m16.752278235s) [upload=4621]: Waiting for error group managing segments... Oct 27 19:56:59.801926071 (5h14m17.151233347s) [noise=true , address=81.171.14.70:4034 , piece_id=QQXBWVXWX24ETMCIFZ7LTONF3LVGL3RLDXXSIQGX62PLWPJSFM5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.811249835 (5h14m17.160557121s) [noise=true , address=81.171.14.70:4048 , piece_id=PR2HDQTP5VKTMF7HG6VNKKUJD66LC2HKQVAZUGEZKKZNKH2QCZFQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.858197924 (5h14m17.20750521s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.897902089 (5h14m17.247209375s) [noise=true , address=89.149.201.147:4023 , piece_id=LEEWZDLYWPSSUDRIPEQK4NKC4PFF6JVLEBPZMQJLWCG4YZHRFQ3A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.901773520 (5h14m17.251080806s) [noise=true , address=89.149.201.147:4010 , piece_id=TRY6HG262YJRDUVNLLHRYG554BVP4J5TWPGHJUVN5N55WHU3C4ZA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902238556 (5h14m17.251545842s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902438327 (5h14m17.251745603s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.917540715 (5h14m17.266847991s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.926674337 (5h14m17.275981623s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.328819374 (5h14m17.67812666s) [noise=true , address=81.171.14.70:4030 , piece_id=ZTVTBK2E7IGYOYZMK6TTHKMMBST52RCZVKM776PV3USLXONMH66Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.349296225 (5h14m17.698603501s) [noise=true , address=89.149.201.147:4011 , piece_id=KQM7L432OU7BYKWL2QHAIKRNPKPA7NOPSZJDOAAFQEGV66PZMGTA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.352514193 (5h14m17.701821479s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.380253194 (5h14m17.72956049s) [noise=true , address=185.167.204.205:4030 , piece_id=35Y5DRDZZHA7LEUZDJQ36RU4MB6DQN2AAIG7E45EIC6YM3MQ7HHQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.385432055 (5h14m17.734739341s) [noise=true , address=185.167.204.205:4023 , piece_id=ACMLGJMZ5H25VH7VULKAIBWIHIAEE2AV366UEZFOJDZRNQ4NGYLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.388082382 (5h14m17.737389678s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.391016018 (5h14m17.740323304s) [noise=true , address=185.167.204.206:4056 , piece_id=G5D6SI6CVQHAVSAPS6GJRVZIQ4YQ23PFH7KL26TVK4I4BBTW2C5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.396014777 (5h14m17.745322053s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.401559615 (5h14m17.750866901s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.414355180 (5h14m17.763662466s) [noise=true , address=185.167.204.206:4041 , piece_id=NUWDUCCNI44ANZLNYISQQKFSBK7WYTPAPGQ6JJ43IP44LNCLT4SA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.421539241 (5h14m17.770846537s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.423334288 (5h14m17.772641574s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.447102799 (5h14m17.796410075s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.473754258 (5h14m17.823061544s) [noise=true , address=185.167.204.206:4066 , piece_id=KKDV46RFAENAUYDYRPDCV64LWX5RB7JVRKB3HY7BXB4LU7DITSLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.481151965 (5h14m17.830459251s) [noise=true , address=185.167.204.205:4019 , piece_id=7LYDTS6D3HQSDMVBCMRLB5NZIRLP44VRSWCUT5EKSGSRPWP4RHIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.486154732 (5h14m17.835462018s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:57.813202781 (5h15m15.162510068s) [noise=true , address=185.167.204.206:4053 , piece_id=UFA7HABPN4LRP7RSCHBHNXT5AMEDFUF3DSYEGWBFSXOTOXRQXT7A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: rpc: dial tcp 185.167.204.206:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813463338 (5h15m15.162770614s) [noise=true , address=185.167.204.206:4047 , piece_id=Z2QX5FHJKEZVW4FXDV2NNUMXEBVPGEW4Z7C3GBVZRUTNFR4MA5IA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: write tcp 188.166.225.170:59506->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813491391 (5h15m15.162798677s) [noise=true , address=185.167.204.205:4013 , piece_id=KICB6KLFDM3KFOGJ52UVXO3G4BUPSSJF657ZIWUIVP56M2SLGMWA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: context deadline exceeded; context deadline exceeded Oct 27 19:57:57.813536527 (5h15m15.162843813s) [seg_pos={0 0}, upload=4621]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Oct 27 19:57:58.016780515 (5h15m15.366087802s) [upload=4621]: Done uploading segments... Oct 27 19:57:58.016879354 (5h15m15.36618663s) [upload=4621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.021611393s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 5.049587717s FAIL (2025-10-27 20:18:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== eg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577374904 (5h14m15.926682211s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577388009 (5h14m15.926695285s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577699713 (5h14m15.927006989s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577834229 (5h14m15.927141505s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578014502 (5h14m15.927321778s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578280880 (5h14m15.927588156s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578608904 (5h14m15.92791618s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578935426 (5h14m15.928242712s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579041648 (5h14m15.928348924s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579222723 (5h14m15.928529999s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579486625 (5h14m15.928793901s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579780545 (5h14m15.929087821s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.580102097 (5h14m15.929409403s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:59.402940250 (5h14m16.752247547s) [upload=4621]: Next returned nil segment Oct 27 19:56:59.402970929 (5h14m16.752278235s) [upload=4621]: Waiting for error group managing segments... Oct 27 19:56:59.801926071 (5h14m17.151233347s) [noise=true , address=81.171.14.70:4034 , piece_id=QQXBWVXWX24ETMCIFZ7LTONF3LVGL3RLDXXSIQGX62PLWPJSFM5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.811249835 (5h14m17.160557121s) [noise=true , address=81.171.14.70:4048 , piece_id=PR2HDQTP5VKTMF7HG6VNKKUJD66LC2HKQVAZUGEZKKZNKH2QCZFQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.858197924 (5h14m17.20750521s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.897902089 (5h14m17.247209375s) [noise=true , address=89.149.201.147:4023 , piece_id=LEEWZDLYWPSSUDRIPEQK4NKC4PFF6JVLEBPZMQJLWCG4YZHRFQ3A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.901773520 (5h14m17.251080806s) [noise=true , address=89.149.201.147:4010 , piece_id=TRY6HG262YJRDUVNLLHRYG554BVP4J5TWPGHJUVN5N55WHU3C4ZA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902238556 (5h14m17.251545842s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902438327 (5h14m17.251745603s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.917540715 (5h14m17.266847991s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.926674337 (5h14m17.275981623s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.328819374 (5h14m17.67812666s) [noise=true , address=81.171.14.70:4030 , piece_id=ZTVTBK2E7IGYOYZMK6TTHKMMBST52RCZVKM776PV3USLXONMH66Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.349296225 (5h14m17.698603501s) [noise=true , address=89.149.201.147:4011 , piece_id=KQM7L432OU7BYKWL2QHAIKRNPKPA7NOPSZJDOAAFQEGV66PZMGTA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.352514193 (5h14m17.701821479s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.380253194 (5h14m17.72956049s) [noise=true , address=185.167.204.205:4030 , piece_id=35Y5DRDZZHA7LEUZDJQ36RU4MB6DQN2AAIG7E45EIC6YM3MQ7HHQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.385432055 (5h14m17.734739341s) [noise=true , address=185.167.204.205:4023 , piece_id=ACMLGJMZ5H25VH7VULKAIBWIHIAEE2AV366UEZFOJDZRNQ4NGYLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.388082382 (5h14m17.737389678s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.391016018 (5h14m17.740323304s) [noise=true , address=185.167.204.206:4056 , piece_id=G5D6SI6CVQHAVSAPS6GJRVZIQ4YQ23PFH7KL26TVK4I4BBTW2C5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.396014777 (5h14m17.745322053s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.401559615 (5h14m17.750866901s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.414355180 (5h14m17.763662466s) [noise=true , address=185.167.204.206:4041 , piece_id=NUWDUCCNI44ANZLNYISQQKFSBK7WYTPAPGQ6JJ43IP44LNCLT4SA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.421539241 (5h14m17.770846537s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.423334288 (5h14m17.772641574s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.447102799 (5h14m17.796410075s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.473754258 (5h14m17.823061544s) [noise=true , address=185.167.204.206:4066 , piece_id=KKDV46RFAENAUYDYRPDCV64LWX5RB7JVRKB3HY7BXB4LU7DITSLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.481151965 (5h14m17.830459251s) [noise=true , address=185.167.204.205:4019 , piece_id=7LYDTS6D3HQSDMVBCMRLB5NZIRLP44VRSWCUT5EKSGSRPWP4RHIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.486154732 (5h14m17.835462018s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:57.813202781 (5h15m15.162510068s) [noise=true , address=185.167.204.206:4053 , piece_id=UFA7HABPN4LRP7RSCHBHNXT5AMEDFUF3DSYEGWBFSXOTOXRQXT7A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: rpc: dial tcp 185.167.204.206:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813463338 (5h15m15.162770614s) [noise=true , address=185.167.204.206:4047 , piece_id=Z2QX5FHJKEZVW4FXDV2NNUMXEBVPGEW4Z7C3GBVZRUTNFR4MA5IA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: write tcp 188.166.225.170:59506->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813491391 (5h15m15.162798677s) [noise=true , address=185.167.204.205:4013 , piece_id=KICB6KLFDM3KFOGJ52UVXO3G4BUPSSJF657ZIWUIVP56M2SLGMWA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: context deadline exceeded; context deadline exceeded Oct 27 19:57:57.813536527 (5h15m15.162843813s) [seg_pos={0 0}, upload=4621]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Oct 27 19:57:58.016780515 (5h15m15.366087802s) [upload=4621]: Done uploading segments... Oct 27 19:57:58.016879354 (5h15m15.36618663s) [upload=4621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.049611542s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 3.456413476s FAIL (2025-10-27 20:18:37 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.456442662s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 4.993973537s FAIL (2025-10-27 20:18:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== eg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577374904 (5h14m15.926682211s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577388009 (5h14m15.926695285s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577699713 (5h14m15.927006989s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577834229 (5h14m15.927141505s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578014502 (5h14m15.927321778s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578280880 (5h14m15.927588156s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578608904 (5h14m15.92791618s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578935426 (5h14m15.928242712s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579041648 (5h14m15.928348924s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579222723 (5h14m15.928529999s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579486625 (5h14m15.928793901s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579780545 (5h14m15.929087821s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.580102097 (5h14m15.929409403s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:59.402940250 (5h14m16.752247547s) [upload=4621]: Next returned nil segment Oct 27 19:56:59.402970929 (5h14m16.752278235s) [upload=4621]: Waiting for error group managing segments... Oct 27 19:56:59.801926071 (5h14m17.151233347s) [noise=true , address=81.171.14.70:4034 , piece_id=QQXBWVXWX24ETMCIFZ7LTONF3LVGL3RLDXXSIQGX62PLWPJSFM5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.811249835 (5h14m17.160557121s) [noise=true , address=81.171.14.70:4048 , piece_id=PR2HDQTP5VKTMF7HG6VNKKUJD66LC2HKQVAZUGEZKKZNKH2QCZFQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.858197924 (5h14m17.20750521s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.897902089 (5h14m17.247209375s) [noise=true , address=89.149.201.147:4023 , piece_id=LEEWZDLYWPSSUDRIPEQK4NKC4PFF6JVLEBPZMQJLWCG4YZHRFQ3A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.901773520 (5h14m17.251080806s) [noise=true , address=89.149.201.147:4010 , piece_id=TRY6HG262YJRDUVNLLHRYG554BVP4J5TWPGHJUVN5N55WHU3C4ZA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902238556 (5h14m17.251545842s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902438327 (5h14m17.251745603s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.917540715 (5h14m17.266847991s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.926674337 (5h14m17.275981623s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.328819374 (5h14m17.67812666s) [noise=true , address=81.171.14.70:4030 , piece_id=ZTVTBK2E7IGYOYZMK6TTHKMMBST52RCZVKM776PV3USLXONMH66Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.349296225 (5h14m17.698603501s) [noise=true , address=89.149.201.147:4011 , piece_id=KQM7L432OU7BYKWL2QHAIKRNPKPA7NOPSZJDOAAFQEGV66PZMGTA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.352514193 (5h14m17.701821479s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.380253194 (5h14m17.72956049s) [noise=true , address=185.167.204.205:4030 , piece_id=35Y5DRDZZHA7LEUZDJQ36RU4MB6DQN2AAIG7E45EIC6YM3MQ7HHQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.385432055 (5h14m17.734739341s) [noise=true , address=185.167.204.205:4023 , piece_id=ACMLGJMZ5H25VH7VULKAIBWIHIAEE2AV366UEZFOJDZRNQ4NGYLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.388082382 (5h14m17.737389678s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.391016018 (5h14m17.740323304s) [noise=true , address=185.167.204.206:4056 , piece_id=G5D6SI6CVQHAVSAPS6GJRVZIQ4YQ23PFH7KL26TVK4I4BBTW2C5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.396014777 (5h14m17.745322053s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.401559615 (5h14m17.750866901s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.414355180 (5h14m17.763662466s) [noise=true , address=185.167.204.206:4041 , piece_id=NUWDUCCNI44ANZLNYISQQKFSBK7WYTPAPGQ6JJ43IP44LNCLT4SA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.421539241 (5h14m17.770846537s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.423334288 (5h14m17.772641574s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.447102799 (5h14m17.796410075s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.473754258 (5h14m17.823061544s) [noise=true , address=185.167.204.206:4066 , piece_id=KKDV46RFAENAUYDYRPDCV64LWX5RB7JVRKB3HY7BXB4LU7DITSLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.481151965 (5h14m17.830459251s) [noise=true , address=185.167.204.205:4019 , piece_id=7LYDTS6D3HQSDMVBCMRLB5NZIRLP44VRSWCUT5EKSGSRPWP4RHIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.486154732 (5h14m17.835462018s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:57.813202781 (5h15m15.162510068s) [noise=true , address=185.167.204.206:4053 , piece_id=UFA7HABPN4LRP7RSCHBHNXT5AMEDFUF3DSYEGWBFSXOTOXRQXT7A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: rpc: dial tcp 185.167.204.206:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813463338 (5h15m15.162770614s) [noise=true , address=185.167.204.206:4047 , piece_id=Z2QX5FHJKEZVW4FXDV2NNUMXEBVPGEW4Z7C3GBVZRUTNFR4MA5IA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: write tcp 188.166.225.170:59506->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813491391 (5h15m15.162798677s) [noise=true , address=185.167.204.205:4013 , piece_id=KICB6KLFDM3KFOGJ52UVXO3G4BUPSSJF657ZIWUIVP56M2SLGMWA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: context deadline exceeded; context deadline exceeded Oct 27 19:57:57.813536527 (5h15m15.162843813s) [seg_pos={0 0}, upload=4621]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Oct 27 19:57:58.016780515 (5h15m15.366087802s) [upload=4621]: Done uploading segments... Oct 27 19:57:58.016879354 (5h15m15.36618663s) [upload=4621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 4.994006249s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 4.989214117s FAIL (2025-10-27 20:18:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== eg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577374904 (5h14m15.926682211s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577388009 (5h14m15.926695285s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577699713 (5h14m15.927006989s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577834229 (5h14m15.927141505s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578014502 (5h14m15.927321778s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578280880 (5h14m15.927588156s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578608904 (5h14m15.92791618s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578935426 (5h14m15.928242712s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579041648 (5h14m15.928348924s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579222723 (5h14m15.928529999s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579486625 (5h14m15.928793901s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579780545 (5h14m15.929087821s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.580102097 (5h14m15.929409403s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:59.402940250 (5h14m16.752247547s) [upload=4621]: Next returned nil segment Oct 27 19:56:59.402970929 (5h14m16.752278235s) [upload=4621]: Waiting for error group managing segments... Oct 27 19:56:59.801926071 (5h14m17.151233347s) [noise=true , address=81.171.14.70:4034 , piece_id=QQXBWVXWX24ETMCIFZ7LTONF3LVGL3RLDXXSIQGX62PLWPJSFM5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.811249835 (5h14m17.160557121s) [noise=true , address=81.171.14.70:4048 , piece_id=PR2HDQTP5VKTMF7HG6VNKKUJD66LC2HKQVAZUGEZKKZNKH2QCZFQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.858197924 (5h14m17.20750521s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.897902089 (5h14m17.247209375s) [noise=true , address=89.149.201.147:4023 , piece_id=LEEWZDLYWPSSUDRIPEQK4NKC4PFF6JVLEBPZMQJLWCG4YZHRFQ3A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.901773520 (5h14m17.251080806s) [noise=true , address=89.149.201.147:4010 , piece_id=TRY6HG262YJRDUVNLLHRYG554BVP4J5TWPGHJUVN5N55WHU3C4ZA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902238556 (5h14m17.251545842s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902438327 (5h14m17.251745603s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.917540715 (5h14m17.266847991s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.926674337 (5h14m17.275981623s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.328819374 (5h14m17.67812666s) [noise=true , address=81.171.14.70:4030 , piece_id=ZTVTBK2E7IGYOYZMK6TTHKMMBST52RCZVKM776PV3USLXONMH66Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.349296225 (5h14m17.698603501s) [noise=true , address=89.149.201.147:4011 , piece_id=KQM7L432OU7BYKWL2QHAIKRNPKPA7NOPSZJDOAAFQEGV66PZMGTA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.352514193 (5h14m17.701821479s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.380253194 (5h14m17.72956049s) [noise=true , address=185.167.204.205:4030 , piece_id=35Y5DRDZZHA7LEUZDJQ36RU4MB6DQN2AAIG7E45EIC6YM3MQ7HHQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.385432055 (5h14m17.734739341s) [noise=true , address=185.167.204.205:4023 , piece_id=ACMLGJMZ5H25VH7VULKAIBWIHIAEE2AV366UEZFOJDZRNQ4NGYLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.388082382 (5h14m17.737389678s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.391016018 (5h14m17.740323304s) [noise=true , address=185.167.204.206:4056 , piece_id=G5D6SI6CVQHAVSAPS6GJRVZIQ4YQ23PFH7KL26TVK4I4BBTW2C5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.396014777 (5h14m17.745322053s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.401559615 (5h14m17.750866901s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.414355180 (5h14m17.763662466s) [noise=true , address=185.167.204.206:4041 , piece_id=NUWDUCCNI44ANZLNYISQQKFSBK7WYTPAPGQ6JJ43IP44LNCLT4SA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.421539241 (5h14m17.770846537s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.423334288 (5h14m17.772641574s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.447102799 (5h14m17.796410075s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.473754258 (5h14m17.823061544s) [noise=true , address=185.167.204.206:4066 , piece_id=KKDV46RFAENAUYDYRPDCV64LWX5RB7JVRKB3HY7BXB4LU7DITSLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.481151965 (5h14m17.830459251s) [noise=true , address=185.167.204.205:4019 , piece_id=7LYDTS6D3HQSDMVBCMRLB5NZIRLP44VRSWCUT5EKSGSRPWP4RHIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.486154732 (5h14m17.835462018s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:57.813202781 (5h15m15.162510068s) [noise=true , address=185.167.204.206:4053 , piece_id=UFA7HABPN4LRP7RSCHBHNXT5AMEDFUF3DSYEGWBFSXOTOXRQXT7A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: rpc: dial tcp 185.167.204.206:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813463338 (5h15m15.162770614s) [noise=true , address=185.167.204.206:4047 , piece_id=Z2QX5FHJKEZVW4FXDV2NNUMXEBVPGEW4Z7C3GBVZRUTNFR4MA5IA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: write tcp 188.166.225.170:59506->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813491391 (5h15m15.162798677s) [noise=true , address=185.167.204.205:4013 , piece_id=KICB6KLFDM3KFOGJ52UVXO3G4BUPSSJF657ZIWUIVP56M2SLGMWA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: context deadline exceeded; context deadline exceeded Oct 27 19:57:57.813536527 (5h15m15.162843813s) [seg_pos={0 0}, upload=4621]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Oct 27 19:57:58.016780515 (5h15m15.366087802s) [upload=4621]: Done uploading segments... Oct 27 19:57:58.016879354 (5h15m15.36618663s) [upload=4621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 4.989249656s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.080301902s FAIL (2025-10-27 20:17:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== eg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577374904 (5h14m15.926682211s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577388009 (5h14m15.926695285s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577699713 (5h14m15.927006989s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577834229 (5h14m15.927141505s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578014502 (5h14m15.927321778s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578280880 (5h14m15.927588156s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578608904 (5h14m15.92791618s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578935426 (5h14m15.928242712s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579041648 (5h14m15.928348924s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579222723 (5h14m15.928529999s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579486625 (5h14m15.928793901s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579780545 (5h14m15.929087821s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.580102097 (5h14m15.929409403s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:59.402940250 (5h14m16.752247547s) [upload=4621]: Next returned nil segment Oct 27 19:56:59.402970929 (5h14m16.752278235s) [upload=4621]: Waiting for error group managing segments... Oct 27 19:56:59.801926071 (5h14m17.151233347s) [noise=true , address=81.171.14.70:4034 , piece_id=QQXBWVXWX24ETMCIFZ7LTONF3LVGL3RLDXXSIQGX62PLWPJSFM5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.811249835 (5h14m17.160557121s) [noise=true , address=81.171.14.70:4048 , piece_id=PR2HDQTP5VKTMF7HG6VNKKUJD66LC2HKQVAZUGEZKKZNKH2QCZFQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.858197924 (5h14m17.20750521s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.897902089 (5h14m17.247209375s) [noise=true , address=89.149.201.147:4023 , piece_id=LEEWZDLYWPSSUDRIPEQK4NKC4PFF6JVLEBPZMQJLWCG4YZHRFQ3A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.901773520 (5h14m17.251080806s) [noise=true , address=89.149.201.147:4010 , piece_id=TRY6HG262YJRDUVNLLHRYG554BVP4J5TWPGHJUVN5N55WHU3C4ZA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902238556 (5h14m17.251545842s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902438327 (5h14m17.251745603s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.917540715 (5h14m17.266847991s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.926674337 (5h14m17.275981623s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.328819374 (5h14m17.67812666s) [noise=true , address=81.171.14.70:4030 , piece_id=ZTVTBK2E7IGYOYZMK6TTHKMMBST52RCZVKM776PV3USLXONMH66Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.349296225 (5h14m17.698603501s) [noise=true , address=89.149.201.147:4011 , piece_id=KQM7L432OU7BYKWL2QHAIKRNPKPA7NOPSZJDOAAFQEGV66PZMGTA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.352514193 (5h14m17.701821479s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.380253194 (5h14m17.72956049s) [noise=true , address=185.167.204.205:4030 , piece_id=35Y5DRDZZHA7LEUZDJQ36RU4MB6DQN2AAIG7E45EIC6YM3MQ7HHQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.385432055 (5h14m17.734739341s) [noise=true , address=185.167.204.205:4023 , piece_id=ACMLGJMZ5H25VH7VULKAIBWIHIAEE2AV366UEZFOJDZRNQ4NGYLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.388082382 (5h14m17.737389678s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.391016018 (5h14m17.740323304s) [noise=true , address=185.167.204.206:4056 , piece_id=G5D6SI6CVQHAVSAPS6GJRVZIQ4YQ23PFH7KL26TVK4I4BBTW2C5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.396014777 (5h14m17.745322053s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.401559615 (5h14m17.750866901s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.414355180 (5h14m17.763662466s) [noise=true , address=185.167.204.206:4041 , piece_id=NUWDUCCNI44ANZLNYISQQKFSBK7WYTPAPGQ6JJ43IP44LNCLT4SA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.421539241 (5h14m17.770846537s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.423334288 (5h14m17.772641574s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.447102799 (5h14m17.796410075s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.473754258 (5h14m17.823061544s) [noise=true , address=185.167.204.206:4066 , piece_id=KKDV46RFAENAUYDYRPDCV64LWX5RB7JVRKB3HY7BXB4LU7DITSLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.481151965 (5h14m17.830459251s) [noise=true , address=185.167.204.205:4019 , piece_id=7LYDTS6D3HQSDMVBCMRLB5NZIRLP44VRSWCUT5EKSGSRPWP4RHIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.486154732 (5h14m17.835462018s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:57.813202781 (5h15m15.162510068s) [noise=true , address=185.167.204.206:4053 , piece_id=UFA7HABPN4LRP7RSCHBHNXT5AMEDFUF3DSYEGWBFSXOTOXRQXT7A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: rpc: dial tcp 185.167.204.206:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813463338 (5h15m15.162770614s) [noise=true , address=185.167.204.206:4047 , piece_id=Z2QX5FHJKEZVW4FXDV2NNUMXEBVPGEW4Z7C3GBVZRUTNFR4MA5IA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: write tcp 188.166.225.170:59506->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813491391 (5h15m15.162798677s) [noise=true , address=185.167.204.205:4013 , piece_id=KICB6KLFDM3KFOGJ52UVXO3G4BUPSSJF657ZIWUIVP56M2SLGMWA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: context deadline exceeded; context deadline exceeded Oct 27 19:57:57.813536527 (5h15m15.162843813s) [seg_pos={0 0}, upload=4621]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Oct 27 19:57:58.016780515 (5h15m15.366087802s) [upload=4621]: Done uploading segments... Oct 27 19:57:58.016879354 (5h15m15.36618663s) [upload=4621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.080330305s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 3.419370375s FAIL (2025-10-27 20:17:24 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.41943096s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 4.958988347s FAIL (2025-10-27 20:17:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== eg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577374904 (5h14m15.926682211s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577388009 (5h14m15.926695285s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577699713 (5h14m15.927006989s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577834229 (5h14m15.927141505s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578014502 (5h14m15.927321778s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578280880 (5h14m15.927588156s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578608904 (5h14m15.92791618s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578935426 (5h14m15.928242712s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579041648 (5h14m15.928348924s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579222723 (5h14m15.928529999s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579486625 (5h14m15.928793901s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579780545 (5h14m15.929087821s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.580102097 (5h14m15.929409403s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:59.402940250 (5h14m16.752247547s) [upload=4621]: Next returned nil segment Oct 27 19:56:59.402970929 (5h14m16.752278235s) [upload=4621]: Waiting for error group managing segments... Oct 27 19:56:59.801926071 (5h14m17.151233347s) [noise=true , address=81.171.14.70:4034 , piece_id=QQXBWVXWX24ETMCIFZ7LTONF3LVGL3RLDXXSIQGX62PLWPJSFM5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.811249835 (5h14m17.160557121s) [noise=true , address=81.171.14.70:4048 , piece_id=PR2HDQTP5VKTMF7HG6VNKKUJD66LC2HKQVAZUGEZKKZNKH2QCZFQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.858197924 (5h14m17.20750521s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.897902089 (5h14m17.247209375s) [noise=true , address=89.149.201.147:4023 , piece_id=LEEWZDLYWPSSUDRIPEQK4NKC4PFF6JVLEBPZMQJLWCG4YZHRFQ3A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.901773520 (5h14m17.251080806s) [noise=true , address=89.149.201.147:4010 , piece_id=TRY6HG262YJRDUVNLLHRYG554BVP4J5TWPGHJUVN5N55WHU3C4ZA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902238556 (5h14m17.251545842s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902438327 (5h14m17.251745603s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.917540715 (5h14m17.266847991s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.926674337 (5h14m17.275981623s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.328819374 (5h14m17.67812666s) [noise=true , address=81.171.14.70:4030 , piece_id=ZTVTBK2E7IGYOYZMK6TTHKMMBST52RCZVKM776PV3USLXONMH66Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.349296225 (5h14m17.698603501s) [noise=true , address=89.149.201.147:4011 , piece_id=KQM7L432OU7BYKWL2QHAIKRNPKPA7NOPSZJDOAAFQEGV66PZMGTA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.352514193 (5h14m17.701821479s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.380253194 (5h14m17.72956049s) [noise=true , address=185.167.204.205:4030 , piece_id=35Y5DRDZZHA7LEUZDJQ36RU4MB6DQN2AAIG7E45EIC6YM3MQ7HHQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.385432055 (5h14m17.734739341s) [noise=true , address=185.167.204.205:4023 , piece_id=ACMLGJMZ5H25VH7VULKAIBWIHIAEE2AV366UEZFOJDZRNQ4NGYLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.388082382 (5h14m17.737389678s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.391016018 (5h14m17.740323304s) [noise=true , address=185.167.204.206:4056 , piece_id=G5D6SI6CVQHAVSAPS6GJRVZIQ4YQ23PFH7KL26TVK4I4BBTW2C5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.396014777 (5h14m17.745322053s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.401559615 (5h14m17.750866901s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.414355180 (5h14m17.763662466s) [noise=true , address=185.167.204.206:4041 , piece_id=NUWDUCCNI44ANZLNYISQQKFSBK7WYTPAPGQ6JJ43IP44LNCLT4SA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.421539241 (5h14m17.770846537s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.423334288 (5h14m17.772641574s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.447102799 (5h14m17.796410075s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.473754258 (5h14m17.823061544s) [noise=true , address=185.167.204.206:4066 , piece_id=KKDV46RFAENAUYDYRPDCV64LWX5RB7JVRKB3HY7BXB4LU7DITSLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.481151965 (5h14m17.830459251s) [noise=true , address=185.167.204.205:4019 , piece_id=7LYDTS6D3HQSDMVBCMRLB5NZIRLP44VRSWCUT5EKSGSRPWP4RHIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.486154732 (5h14m17.835462018s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:57.813202781 (5h15m15.162510068s) [noise=true , address=185.167.204.206:4053 , piece_id=UFA7HABPN4LRP7RSCHBHNXT5AMEDFUF3DSYEGWBFSXOTOXRQXT7A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: rpc: dial tcp 185.167.204.206:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813463338 (5h15m15.162770614s) [noise=true , address=185.167.204.206:4047 , piece_id=Z2QX5FHJKEZVW4FXDV2NNUMXEBVPGEW4Z7C3GBVZRUTNFR4MA5IA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: write tcp 188.166.225.170:59506->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813491391 (5h15m15.162798677s) [noise=true , address=185.167.204.205:4013 , piece_id=KICB6KLFDM3KFOGJ52UVXO3G4BUPSSJF657ZIWUIVP56M2SLGMWA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: context deadline exceeded; context deadline exceeded Oct 27 19:57:57.813536527 (5h15m15.162843813s) [seg_pos={0 0}, upload=4621]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Oct 27 19:57:58.016780515 (5h15m15.366087802s) [upload=4621]: Done uploading segments... Oct 27 19:57:58.016879354 (5h15m15.36618663s) [upload=4621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 4.959023143s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 3.476064734s FAIL (2025-10-27 20:16:32 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.476089362s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [X.......................................................X...] duration: 3.385020357s FAIL (2025-10-27 20:16:06 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.385051697s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.006414968s FAIL (2025-10-27 20:15:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== eg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577374904 (5h14m15.926682211s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577388009 (5h14m15.926695285s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577699713 (5h14m15.927006989s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577834229 (5h14m15.927141505s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578014502 (5h14m15.927321778s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578280880 (5h14m15.927588156s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578608904 (5h14m15.92791618s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578935426 (5h14m15.928242712s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579041648 (5h14m15.928348924s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579222723 (5h14m15.928529999s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579486625 (5h14m15.928793901s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579780545 (5h14m15.929087821s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.580102097 (5h14m15.929409403s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:59.402940250 (5h14m16.752247547s) [upload=4621]: Next returned nil segment Oct 27 19:56:59.402970929 (5h14m16.752278235s) [upload=4621]: Waiting for error group managing segments... Oct 27 19:56:59.801926071 (5h14m17.151233347s) [noise=true , address=81.171.14.70:4034 , piece_id=QQXBWVXWX24ETMCIFZ7LTONF3LVGL3RLDXXSIQGX62PLWPJSFM5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.811249835 (5h14m17.160557121s) [noise=true , address=81.171.14.70:4048 , piece_id=PR2HDQTP5VKTMF7HG6VNKKUJD66LC2HKQVAZUGEZKKZNKH2QCZFQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.858197924 (5h14m17.20750521s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.897902089 (5h14m17.247209375s) [noise=true , address=89.149.201.147:4023 , piece_id=LEEWZDLYWPSSUDRIPEQK4NKC4PFF6JVLEBPZMQJLWCG4YZHRFQ3A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.901773520 (5h14m17.251080806s) [noise=true , address=89.149.201.147:4010 , piece_id=TRY6HG262YJRDUVNLLHRYG554BVP4J5TWPGHJUVN5N55WHU3C4ZA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902238556 (5h14m17.251545842s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902438327 (5h14m17.251745603s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.917540715 (5h14m17.266847991s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.926674337 (5h14m17.275981623s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.328819374 (5h14m17.67812666s) [noise=true , address=81.171.14.70:4030 , piece_id=ZTVTBK2E7IGYOYZMK6TTHKMMBST52RCZVKM776PV3USLXONMH66Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.349296225 (5h14m17.698603501s) [noise=true , address=89.149.201.147:4011 , piece_id=KQM7L432OU7BYKWL2QHAIKRNPKPA7NOPSZJDOAAFQEGV66PZMGTA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.352514193 (5h14m17.701821479s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.380253194 (5h14m17.72956049s) [noise=true , address=185.167.204.205:4030 , piece_id=35Y5DRDZZHA7LEUZDJQ36RU4MB6DQN2AAIG7E45EIC6YM3MQ7HHQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.385432055 (5h14m17.734739341s) [noise=true , address=185.167.204.205:4023 , piece_id=ACMLGJMZ5H25VH7VULKAIBWIHIAEE2AV366UEZFOJDZRNQ4NGYLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.388082382 (5h14m17.737389678s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.391016018 (5h14m17.740323304s) [noise=true , address=185.167.204.206:4056 , piece_id=G5D6SI6CVQHAVSAPS6GJRVZIQ4YQ23PFH7KL26TVK4I4BBTW2C5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.396014777 (5h14m17.745322053s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.401559615 (5h14m17.750866901s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.414355180 (5h14m17.763662466s) [noise=true , address=185.167.204.206:4041 , piece_id=NUWDUCCNI44ANZLNYISQQKFSBK7WYTPAPGQ6JJ43IP44LNCLT4SA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.421539241 (5h14m17.770846537s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.423334288 (5h14m17.772641574s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.447102799 (5h14m17.796410075s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.473754258 (5h14m17.823061544s) [noise=true , address=185.167.204.206:4066 , piece_id=KKDV46RFAENAUYDYRPDCV64LWX5RB7JVRKB3HY7BXB4LU7DITSLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.481151965 (5h14m17.830459251s) [noise=true , address=185.167.204.205:4019 , piece_id=7LYDTS6D3HQSDMVBCMRLB5NZIRLP44VRSWCUT5EKSGSRPWP4RHIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.486154732 (5h14m17.835462018s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:57.813202781 (5h15m15.162510068s) [noise=true , address=185.167.204.206:4053 , piece_id=UFA7HABPN4LRP7RSCHBHNXT5AMEDFUF3DSYEGWBFSXOTOXRQXT7A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: rpc: dial tcp 185.167.204.206:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813463338 (5h15m15.162770614s) [noise=true , address=185.167.204.206:4047 , piece_id=Z2QX5FHJKEZVW4FXDV2NNUMXEBVPGEW4Z7C3GBVZRUTNFR4MA5IA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: write tcp 188.166.225.170:59506->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813491391 (5h15m15.162798677s) [noise=true , address=185.167.204.205:4013 , piece_id=KICB6KLFDM3KFOGJ52UVXO3G4BUPSSJF657ZIWUIVP56M2SLGMWA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: context deadline exceeded; context deadline exceeded Oct 27 19:57:57.813536527 (5h15m15.162843813s) [seg_pos={0 0}, upload=4621]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Oct 27 19:57:58.016780515 (5h15m15.366087802s) [upload=4621]: Done uploading segments... Oct 27 19:57:58.016879354 (5h15m15.36618663s) [upload=4621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.006446768s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [.X.......................................................X..] duration: 3.570791487s FAIL (2025-10-27 20:15:41 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.570818778s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 4.99643082s FAIL (2025-10-27 20:15:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== eg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577374904 (5h14m15.926682211s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577388009 (5h14m15.926695285s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577699713 (5h14m15.927006989s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577834229 (5h14m15.927141505s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578014502 (5h14m15.927321778s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578280880 (5h14m15.927588156s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578608904 (5h14m15.92791618s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578935426 (5h14m15.928242712s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579041648 (5h14m15.928348924s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579222723 (5h14m15.928529999s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579486625 (5h14m15.928793901s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579780545 (5h14m15.929087821s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.580102097 (5h14m15.929409403s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:59.402940250 (5h14m16.752247547s) [upload=4621]: Next returned nil segment Oct 27 19:56:59.402970929 (5h14m16.752278235s) [upload=4621]: Waiting for error group managing segments... Oct 27 19:56:59.801926071 (5h14m17.151233347s) [noise=true , address=81.171.14.70:4034 , piece_id=QQXBWVXWX24ETMCIFZ7LTONF3LVGL3RLDXXSIQGX62PLWPJSFM5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.811249835 (5h14m17.160557121s) [noise=true , address=81.171.14.70:4048 , piece_id=PR2HDQTP5VKTMF7HG6VNKKUJD66LC2HKQVAZUGEZKKZNKH2QCZFQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.858197924 (5h14m17.20750521s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.897902089 (5h14m17.247209375s) [noise=true , address=89.149.201.147:4023 , piece_id=LEEWZDLYWPSSUDRIPEQK4NKC4PFF6JVLEBPZMQJLWCG4YZHRFQ3A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.901773520 (5h14m17.251080806s) [noise=true , address=89.149.201.147:4010 , piece_id=TRY6HG262YJRDUVNLLHRYG554BVP4J5TWPGHJUVN5N55WHU3C4ZA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902238556 (5h14m17.251545842s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902438327 (5h14m17.251745603s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.917540715 (5h14m17.266847991s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.926674337 (5h14m17.275981623s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.328819374 (5h14m17.67812666s) [noise=true , address=81.171.14.70:4030 , piece_id=ZTVTBK2E7IGYOYZMK6TTHKMMBST52RCZVKM776PV3USLXONMH66Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.349296225 (5h14m17.698603501s) [noise=true , address=89.149.201.147:4011 , piece_id=KQM7L432OU7BYKWL2QHAIKRNPKPA7NOPSZJDOAAFQEGV66PZMGTA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.352514193 (5h14m17.701821479s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.380253194 (5h14m17.72956049s) [noise=true , address=185.167.204.205:4030 , piece_id=35Y5DRDZZHA7LEUZDJQ36RU4MB6DQN2AAIG7E45EIC6YM3MQ7HHQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.385432055 (5h14m17.734739341s) [noise=true , address=185.167.204.205:4023 , piece_id=ACMLGJMZ5H25VH7VULKAIBWIHIAEE2AV366UEZFOJDZRNQ4NGYLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.388082382 (5h14m17.737389678s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.391016018 (5h14m17.740323304s) [noise=true , address=185.167.204.206:4056 , piece_id=G5D6SI6CVQHAVSAPS6GJRVZIQ4YQ23PFH7KL26TVK4I4BBTW2C5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.396014777 (5h14m17.745322053s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.401559615 (5h14m17.750866901s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.414355180 (5h14m17.763662466s) [noise=true , address=185.167.204.206:4041 , piece_id=NUWDUCCNI44ANZLNYISQQKFSBK7WYTPAPGQ6JJ43IP44LNCLT4SA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.421539241 (5h14m17.770846537s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.423334288 (5h14m17.772641574s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.447102799 (5h14m17.796410075s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.473754258 (5h14m17.823061544s) [noise=true , address=185.167.204.206:4066 , piece_id=KKDV46RFAENAUYDYRPDCV64LWX5RB7JVRKB3HY7BXB4LU7DITSLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.481151965 (5h14m17.830459251s) [noise=true , address=185.167.204.205:4019 , piece_id=7LYDTS6D3HQSDMVBCMRLB5NZIRLP44VRSWCUT5EKSGSRPWP4RHIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.486154732 (5h14m17.835462018s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:57.813202781 (5h15m15.162510068s) [noise=true , address=185.167.204.206:4053 , piece_id=UFA7HABPN4LRP7RSCHBHNXT5AMEDFUF3DSYEGWBFSXOTOXRQXT7A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: rpc: dial tcp 185.167.204.206:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813463338 (5h15m15.162770614s) [noise=true , address=185.167.204.206:4047 , piece_id=Z2QX5FHJKEZVW4FXDV2NNUMXEBVPGEW4Z7C3GBVZRUTNFR4MA5IA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: write tcp 188.166.225.170:59506->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813491391 (5h15m15.162798677s) [noise=true , address=185.167.204.205:4013 , piece_id=KICB6KLFDM3KFOGJ52UVXO3G4BUPSSJF657ZIWUIVP56M2SLGMWA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: context deadline exceeded; context deadline exceeded Oct 27 19:57:57.813536527 (5h15m15.162843813s) [seg_pos={0 0}, upload=4621]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Oct 27 19:57:58.016780515 (5h15m15.366087802s) [upload=4621]: Done uploading segments... Oct 27 19:57:58.016879354 (5h15m15.36618663s) [upload=4621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 4.996457581s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.054525958s FAIL (2025-10-27 20:15:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== eg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577374904 (5h14m15.926682211s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577388009 (5h14m15.926695285s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577699713 (5h14m15.927006989s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577834229 (5h14m15.927141505s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578014502 (5h14m15.927321778s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578280880 (5h14m15.927588156s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578608904 (5h14m15.92791618s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578935426 (5h14m15.928242712s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579041648 (5h14m15.928348924s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579222723 (5h14m15.928529999s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579486625 (5h14m15.928793901s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579780545 (5h14m15.929087821s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.580102097 (5h14m15.929409403s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:59.402940250 (5h14m16.752247547s) [upload=4621]: Next returned nil segment Oct 27 19:56:59.402970929 (5h14m16.752278235s) [upload=4621]: Waiting for error group managing segments... Oct 27 19:56:59.801926071 (5h14m17.151233347s) [noise=true , address=81.171.14.70:4034 , piece_id=QQXBWVXWX24ETMCIFZ7LTONF3LVGL3RLDXXSIQGX62PLWPJSFM5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.811249835 (5h14m17.160557121s) [noise=true , address=81.171.14.70:4048 , piece_id=PR2HDQTP5VKTMF7HG6VNKKUJD66LC2HKQVAZUGEZKKZNKH2QCZFQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.858197924 (5h14m17.20750521s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.897902089 (5h14m17.247209375s) [noise=true , address=89.149.201.147:4023 , piece_id=LEEWZDLYWPSSUDRIPEQK4NKC4PFF6JVLEBPZMQJLWCG4YZHRFQ3A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.901773520 (5h14m17.251080806s) [noise=true , address=89.149.201.147:4010 , piece_id=TRY6HG262YJRDUVNLLHRYG554BVP4J5TWPGHJUVN5N55WHU3C4ZA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902238556 (5h14m17.251545842s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902438327 (5h14m17.251745603s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.917540715 (5h14m17.266847991s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.926674337 (5h14m17.275981623s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.328819374 (5h14m17.67812666s) [noise=true , address=81.171.14.70:4030 , piece_id=ZTVTBK2E7IGYOYZMK6TTHKMMBST52RCZVKM776PV3USLXONMH66Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.349296225 (5h14m17.698603501s) [noise=true , address=89.149.201.147:4011 , piece_id=KQM7L432OU7BYKWL2QHAIKRNPKPA7NOPSZJDOAAFQEGV66PZMGTA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.352514193 (5h14m17.701821479s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.380253194 (5h14m17.72956049s) [noise=true , address=185.167.204.205:4030 , piece_id=35Y5DRDZZHA7LEUZDJQ36RU4MB6DQN2AAIG7E45EIC6YM3MQ7HHQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.385432055 (5h14m17.734739341s) [noise=true , address=185.167.204.205:4023 , piece_id=ACMLGJMZ5H25VH7VULKAIBWIHIAEE2AV366UEZFOJDZRNQ4NGYLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.388082382 (5h14m17.737389678s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.391016018 (5h14m17.740323304s) [noise=true , address=185.167.204.206:4056 , piece_id=G5D6SI6CVQHAVSAPS6GJRVZIQ4YQ23PFH7KL26TVK4I4BBTW2C5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.396014777 (5h14m17.745322053s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.401559615 (5h14m17.750866901s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.414355180 (5h14m17.763662466s) [noise=true , address=185.167.204.206:4041 , piece_id=NUWDUCCNI44ANZLNYISQQKFSBK7WYTPAPGQ6JJ43IP44LNCLT4SA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.421539241 (5h14m17.770846537s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.423334288 (5h14m17.772641574s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.447102799 (5h14m17.796410075s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.473754258 (5h14m17.823061544s) [noise=true , address=185.167.204.206:4066 , piece_id=KKDV46RFAENAUYDYRPDCV64LWX5RB7JVRKB3HY7BXB4LU7DITSLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.481151965 (5h14m17.830459251s) [noise=true , address=185.167.204.205:4019 , piece_id=7LYDTS6D3HQSDMVBCMRLB5NZIRLP44VRSWCUT5EKSGSRPWP4RHIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.486154732 (5h14m17.835462018s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:57.813202781 (5h15m15.162510068s) [noise=true , address=185.167.204.206:4053 , piece_id=UFA7HABPN4LRP7RSCHBHNXT5AMEDFUF3DSYEGWBFSXOTOXRQXT7A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: rpc: dial tcp 185.167.204.206:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813463338 (5h15m15.162770614s) [noise=true , address=185.167.204.206:4047 , piece_id=Z2QX5FHJKEZVW4FXDV2NNUMXEBVPGEW4Z7C3GBVZRUTNFR4MA5IA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: write tcp 188.166.225.170:59506->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813491391 (5h15m15.162798677s) [noise=true , address=185.167.204.205:4013 , piece_id=KICB6KLFDM3KFOGJ52UVXO3G4BUPSSJF657ZIWUIVP56M2SLGMWA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: context deadline exceeded; context deadline exceeded Oct 27 19:57:57.813536527 (5h15m15.162843813s) [seg_pos={0 0}, upload=4621]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Oct 27 19:57:58.016780515 (5h15m15.366087802s) [upload=4621]: Done uploading segments... Oct 27 19:57:58.016879354 (5h15m15.36618663s) [upload=4621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.054562939s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=3) [..X.......................................................X.] duration: 3.422905067s FAIL (2025-10-27 20:14:39 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.422930104s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 4.959037961s FAIL (2025-10-27 20:14:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== eg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577374904 (5h14m15.926682211s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577388009 (5h14m15.926695285s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577699713 (5h14m15.927006989s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577834229 (5h14m15.927141505s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578014502 (5h14m15.927321778s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578280880 (5h14m15.927588156s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578608904 (5h14m15.92791618s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578935426 (5h14m15.928242712s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579041648 (5h14m15.928348924s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579222723 (5h14m15.928529999s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579486625 (5h14m15.928793901s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579780545 (5h14m15.929087821s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.580102097 (5h14m15.929409403s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:59.402940250 (5h14m16.752247547s) [upload=4621]: Next returned nil segment Oct 27 19:56:59.402970929 (5h14m16.752278235s) [upload=4621]: Waiting for error group managing segments... Oct 27 19:56:59.801926071 (5h14m17.151233347s) [noise=true , address=81.171.14.70:4034 , piece_id=QQXBWVXWX24ETMCIFZ7LTONF3LVGL3RLDXXSIQGX62PLWPJSFM5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.811249835 (5h14m17.160557121s) [noise=true , address=81.171.14.70:4048 , piece_id=PR2HDQTP5VKTMF7HG6VNKKUJD66LC2HKQVAZUGEZKKZNKH2QCZFQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.858197924 (5h14m17.20750521s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.897902089 (5h14m17.247209375s) [noise=true , address=89.149.201.147:4023 , piece_id=LEEWZDLYWPSSUDRIPEQK4NKC4PFF6JVLEBPZMQJLWCG4YZHRFQ3A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.901773520 (5h14m17.251080806s) [noise=true , address=89.149.201.147:4010 , piece_id=TRY6HG262YJRDUVNLLHRYG554BVP4J5TWPGHJUVN5N55WHU3C4ZA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902238556 (5h14m17.251545842s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902438327 (5h14m17.251745603s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.917540715 (5h14m17.266847991s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.926674337 (5h14m17.275981623s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.328819374 (5h14m17.67812666s) [noise=true , address=81.171.14.70:4030 , piece_id=ZTVTBK2E7IGYOYZMK6TTHKMMBST52RCZVKM776PV3USLXONMH66Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.349296225 (5h14m17.698603501s) [noise=true , address=89.149.201.147:4011 , piece_id=KQM7L432OU7BYKWL2QHAIKRNPKPA7NOPSZJDOAAFQEGV66PZMGTA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.352514193 (5h14m17.701821479s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.380253194 (5h14m17.72956049s) [noise=true , address=185.167.204.205:4030 , piece_id=35Y5DRDZZHA7LEUZDJQ36RU4MB6DQN2AAIG7E45EIC6YM3MQ7HHQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.385432055 (5h14m17.734739341s) [noise=true , address=185.167.204.205:4023 , piece_id=ACMLGJMZ5H25VH7VULKAIBWIHIAEE2AV366UEZFOJDZRNQ4NGYLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.388082382 (5h14m17.737389678s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.391016018 (5h14m17.740323304s) [noise=true , address=185.167.204.206:4056 , piece_id=G5D6SI6CVQHAVSAPS6GJRVZIQ4YQ23PFH7KL26TVK4I4BBTW2C5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.396014777 (5h14m17.745322053s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.401559615 (5h14m17.750866901s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.414355180 (5h14m17.763662466s) [noise=true , address=185.167.204.206:4041 , piece_id=NUWDUCCNI44ANZLNYISQQKFSBK7WYTPAPGQ6JJ43IP44LNCLT4SA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.421539241 (5h14m17.770846537s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.423334288 (5h14m17.772641574s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.447102799 (5h14m17.796410075s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.473754258 (5h14m17.823061544s) [noise=true , address=185.167.204.206:4066 , piece_id=KKDV46RFAENAUYDYRPDCV64LWX5RB7JVRKB3HY7BXB4LU7DITSLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.481151965 (5h14m17.830459251s) [noise=true , address=185.167.204.205:4019 , piece_id=7LYDTS6D3HQSDMVBCMRLB5NZIRLP44VRSWCUT5EKSGSRPWP4RHIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.486154732 (5h14m17.835462018s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:57.813202781 (5h15m15.162510068s) [noise=true , address=185.167.204.206:4053 , piece_id=UFA7HABPN4LRP7RSCHBHNXT5AMEDFUF3DSYEGWBFSXOTOXRQXT7A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: rpc: dial tcp 185.167.204.206:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813463338 (5h15m15.162770614s) [noise=true , address=185.167.204.206:4047 , piece_id=Z2QX5FHJKEZVW4FXDV2NNUMXEBVPGEW4Z7C3GBVZRUTNFR4MA5IA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: write tcp 188.166.225.170:59506->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813491391 (5h15m15.162798677s) [noise=true , address=185.167.204.205:4013 , piece_id=KICB6KLFDM3KFOGJ52UVXO3G4BUPSSJF657ZIWUIVP56M2SLGMWA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: context deadline exceeded; context deadline exceeded Oct 27 19:57:57.813536527 (5h15m15.162843813s) [seg_pos={0 0}, upload=4621]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Oct 27 19:57:58.016780515 (5h15m15.366087802s) [upload=4621]: Done uploading segments... Oct 27 19:57:58.016879354 (5h15m15.36618663s) [upload=4621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 4.959075773s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.031058408s FAIL (2025-10-27 20:13:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== eg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577374904 (5h14m15.926682211s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577388009 (5h14m15.926695285s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577699713 (5h14m15.927006989s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577834229 (5h14m15.927141505s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578014502 (5h14m15.927321778s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578280880 (5h14m15.927588156s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578608904 (5h14m15.92791618s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578935426 (5h14m15.928242712s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579041648 (5h14m15.928348924s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579222723 (5h14m15.928529999s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579486625 (5h14m15.928793901s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579780545 (5h14m15.929087821s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.580102097 (5h14m15.929409403s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:59.402940250 (5h14m16.752247547s) [upload=4621]: Next returned nil segment Oct 27 19:56:59.402970929 (5h14m16.752278235s) [upload=4621]: Waiting for error group managing segments... Oct 27 19:56:59.801926071 (5h14m17.151233347s) [noise=true , address=81.171.14.70:4034 , piece_id=QQXBWVXWX24ETMCIFZ7LTONF3LVGL3RLDXXSIQGX62PLWPJSFM5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.811249835 (5h14m17.160557121s) [noise=true , address=81.171.14.70:4048 , piece_id=PR2HDQTP5VKTMF7HG6VNKKUJD66LC2HKQVAZUGEZKKZNKH2QCZFQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.858197924 (5h14m17.20750521s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.897902089 (5h14m17.247209375s) [noise=true , address=89.149.201.147:4023 , piece_id=LEEWZDLYWPSSUDRIPEQK4NKC4PFF6JVLEBPZMQJLWCG4YZHRFQ3A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.901773520 (5h14m17.251080806s) [noise=true , address=89.149.201.147:4010 , piece_id=TRY6HG262YJRDUVNLLHRYG554BVP4J5TWPGHJUVN5N55WHU3C4ZA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902238556 (5h14m17.251545842s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902438327 (5h14m17.251745603s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.917540715 (5h14m17.266847991s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.926674337 (5h14m17.275981623s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.328819374 (5h14m17.67812666s) [noise=true , address=81.171.14.70:4030 , piece_id=ZTVTBK2E7IGYOYZMK6TTHKMMBST52RCZVKM776PV3USLXONMH66Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.349296225 (5h14m17.698603501s) [noise=true , address=89.149.201.147:4011 , piece_id=KQM7L432OU7BYKWL2QHAIKRNPKPA7NOPSZJDOAAFQEGV66PZMGTA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.352514193 (5h14m17.701821479s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.380253194 (5h14m17.72956049s) [noise=true , address=185.167.204.205:4030 , piece_id=35Y5DRDZZHA7LEUZDJQ36RU4MB6DQN2AAIG7E45EIC6YM3MQ7HHQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.385432055 (5h14m17.734739341s) [noise=true , address=185.167.204.205:4023 , piece_id=ACMLGJMZ5H25VH7VULKAIBWIHIAEE2AV366UEZFOJDZRNQ4NGYLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.388082382 (5h14m17.737389678s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.391016018 (5h14m17.740323304s) [noise=true , address=185.167.204.206:4056 , piece_id=G5D6SI6CVQHAVSAPS6GJRVZIQ4YQ23PFH7KL26TVK4I4BBTW2C5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.396014777 (5h14m17.745322053s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.401559615 (5h14m17.750866901s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.414355180 (5h14m17.763662466s) [noise=true , address=185.167.204.206:4041 , piece_id=NUWDUCCNI44ANZLNYISQQKFSBK7WYTPAPGQ6JJ43IP44LNCLT4SA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.421539241 (5h14m17.770846537s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.423334288 (5h14m17.772641574s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.447102799 (5h14m17.796410075s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.473754258 (5h14m17.823061544s) [noise=true , address=185.167.204.206:4066 , piece_id=KKDV46RFAENAUYDYRPDCV64LWX5RB7JVRKB3HY7BXB4LU7DITSLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.481151965 (5h14m17.830459251s) [noise=true , address=185.167.204.205:4019 , piece_id=7LYDTS6D3HQSDMVBCMRLB5NZIRLP44VRSWCUT5EKSGSRPWP4RHIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.486154732 (5h14m17.835462018s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:57.813202781 (5h15m15.162510068s) [noise=true , address=185.167.204.206:4053 , piece_id=UFA7HABPN4LRP7RSCHBHNXT5AMEDFUF3DSYEGWBFSXOTOXRQXT7A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: rpc: dial tcp 185.167.204.206:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813463338 (5h15m15.162770614s) [noise=true , address=185.167.204.206:4047 , piece_id=Z2QX5FHJKEZVW4FXDV2NNUMXEBVPGEW4Z7C3GBVZRUTNFR4MA5IA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: write tcp 188.166.225.170:59506->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813491391 (5h15m15.162798677s) [noise=true , address=185.167.204.205:4013 , piece_id=KICB6KLFDM3KFOGJ52UVXO3G4BUPSSJF657ZIWUIVP56M2SLGMWA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: context deadline exceeded; context deadline exceeded Oct 27 19:57:57.813536527 (5h15m15.162843813s) [seg_pos={0 0}, upload=4621]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Oct 27 19:57:58.016780515 (5h15m15.366087802s) [upload=4621]: Done uploading segments... Oct 27 19:57:58.016879354 (5h15m15.36618663s) [upload=4621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.031097591s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=3) [X..X.......................................................X] duration: 5.001638812s FAIL (2025-10-27 20:13:30 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001667777s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 4.922074214s FAIL (2025-10-27 20:12:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== eg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577374904 (5h14m15.926682211s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577388009 (5h14m15.926695285s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577699713 (5h14m15.927006989s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577834229 (5h14m15.927141505s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578014502 (5h14m15.927321778s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578280880 (5h14m15.927588156s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578608904 (5h14m15.92791618s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578935426 (5h14m15.928242712s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579041648 (5h14m15.928348924s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579222723 (5h14m15.928529999s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579486625 (5h14m15.928793901s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579780545 (5h14m15.929087821s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.580102097 (5h14m15.929409403s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:59.402940250 (5h14m16.752247547s) [upload=4621]: Next returned nil segment Oct 27 19:56:59.402970929 (5h14m16.752278235s) [upload=4621]: Waiting for error group managing segments... Oct 27 19:56:59.801926071 (5h14m17.151233347s) [noise=true , address=81.171.14.70:4034 , piece_id=QQXBWVXWX24ETMCIFZ7LTONF3LVGL3RLDXXSIQGX62PLWPJSFM5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.811249835 (5h14m17.160557121s) [noise=true , address=81.171.14.70:4048 , piece_id=PR2HDQTP5VKTMF7HG6VNKKUJD66LC2HKQVAZUGEZKKZNKH2QCZFQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.858197924 (5h14m17.20750521s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.897902089 (5h14m17.247209375s) [noise=true , address=89.149.201.147:4023 , piece_id=LEEWZDLYWPSSUDRIPEQK4NKC4PFF6JVLEBPZMQJLWCG4YZHRFQ3A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.901773520 (5h14m17.251080806s) [noise=true , address=89.149.201.147:4010 , piece_id=TRY6HG262YJRDUVNLLHRYG554BVP4J5TWPGHJUVN5N55WHU3C4ZA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902238556 (5h14m17.251545842s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902438327 (5h14m17.251745603s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.917540715 (5h14m17.266847991s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.926674337 (5h14m17.275981623s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.328819374 (5h14m17.67812666s) [noise=true , address=81.171.14.70:4030 , piece_id=ZTVTBK2E7IGYOYZMK6TTHKMMBST52RCZVKM776PV3USLXONMH66Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.349296225 (5h14m17.698603501s) [noise=true , address=89.149.201.147:4011 , piece_id=KQM7L432OU7BYKWL2QHAIKRNPKPA7NOPSZJDOAAFQEGV66PZMGTA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.352514193 (5h14m17.701821479s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.380253194 (5h14m17.72956049s) [noise=true , address=185.167.204.205:4030 , piece_id=35Y5DRDZZHA7LEUZDJQ36RU4MB6DQN2AAIG7E45EIC6YM3MQ7HHQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.385432055 (5h14m17.734739341s) [noise=true , address=185.167.204.205:4023 , piece_id=ACMLGJMZ5H25VH7VULKAIBWIHIAEE2AV366UEZFOJDZRNQ4NGYLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.388082382 (5h14m17.737389678s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.391016018 (5h14m17.740323304s) [noise=true , address=185.167.204.206:4056 , piece_id=G5D6SI6CVQHAVSAPS6GJRVZIQ4YQ23PFH7KL26TVK4I4BBTW2C5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.396014777 (5h14m17.745322053s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.401559615 (5h14m17.750866901s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.414355180 (5h14m17.763662466s) [noise=true , address=185.167.204.206:4041 , piece_id=NUWDUCCNI44ANZLNYISQQKFSBK7WYTPAPGQ6JJ43IP44LNCLT4SA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.421539241 (5h14m17.770846537s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.423334288 (5h14m17.772641574s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.447102799 (5h14m17.796410075s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.473754258 (5h14m17.823061544s) [noise=true , address=185.167.204.206:4066 , piece_id=KKDV46RFAENAUYDYRPDCV64LWX5RB7JVRKB3HY7BXB4LU7DITSLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.481151965 (5h14m17.830459251s) [noise=true , address=185.167.204.205:4019 , piece_id=7LYDTS6D3HQSDMVBCMRLB5NZIRLP44VRSWCUT5EKSGSRPWP4RHIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.486154732 (5h14m17.835462018s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:57.813202781 (5h15m15.162510068s) [noise=true , address=185.167.204.206:4053 , piece_id=UFA7HABPN4LRP7RSCHBHNXT5AMEDFUF3DSYEGWBFSXOTOXRQXT7A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: rpc: dial tcp 185.167.204.206:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813463338 (5h15m15.162770614s) [noise=true , address=185.167.204.206:4047 , piece_id=Z2QX5FHJKEZVW4FXDV2NNUMXEBVPGEW4Z7C3GBVZRUTNFR4MA5IA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: write tcp 188.166.225.170:59506->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813491391 (5h15m15.162798677s) [noise=true , address=185.167.204.205:4013 , piece_id=KICB6KLFDM3KFOGJ52UVXO3G4BUPSSJF657ZIWUIVP56M2SLGMWA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: context deadline exceeded; context deadline exceeded Oct 27 19:57:57.813536527 (5h15m15.162843813s) [seg_pos={0 0}, upload=4621]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Oct 27 19:57:58.016780515 (5h15m15.366087802s) [upload=4621]: Done uploading segments... Oct 27 19:57:58.016879354 (5h15m15.36618663s) [upload=4621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 4.922118127s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=3) [XX..X.......................................................] duration: 4.302247829s FAIL (2025-10-27 20:12:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.302272698s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.095774956s FAIL (2025-10-27 20:11:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== eg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577374904 (5h14m15.926682211s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577388009 (5h14m15.926695285s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577699713 (5h14m15.927006989s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577834229 (5h14m15.927141505s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578014502 (5h14m15.927321778s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578280880 (5h14m15.927588156s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578608904 (5h14m15.92791618s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578935426 (5h14m15.928242712s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579041648 (5h14m15.928348924s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579222723 (5h14m15.928529999s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579486625 (5h14m15.928793901s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579780545 (5h14m15.929087821s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.580102097 (5h14m15.929409403s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:59.402940250 (5h14m16.752247547s) [upload=4621]: Next returned nil segment Oct 27 19:56:59.402970929 (5h14m16.752278235s) [upload=4621]: Waiting for error group managing segments... Oct 27 19:56:59.801926071 (5h14m17.151233347s) [noise=true , address=81.171.14.70:4034 , piece_id=QQXBWVXWX24ETMCIFZ7LTONF3LVGL3RLDXXSIQGX62PLWPJSFM5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.811249835 (5h14m17.160557121s) [noise=true , address=81.171.14.70:4048 , piece_id=PR2HDQTP5VKTMF7HG6VNKKUJD66LC2HKQVAZUGEZKKZNKH2QCZFQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.858197924 (5h14m17.20750521s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.897902089 (5h14m17.247209375s) [noise=true , address=89.149.201.147:4023 , piece_id=LEEWZDLYWPSSUDRIPEQK4NKC4PFF6JVLEBPZMQJLWCG4YZHRFQ3A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.901773520 (5h14m17.251080806s) [noise=true , address=89.149.201.147:4010 , piece_id=TRY6HG262YJRDUVNLLHRYG554BVP4J5TWPGHJUVN5N55WHU3C4ZA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902238556 (5h14m17.251545842s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902438327 (5h14m17.251745603s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.917540715 (5h14m17.266847991s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.926674337 (5h14m17.275981623s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.328819374 (5h14m17.67812666s) [noise=true , address=81.171.14.70:4030 , piece_id=ZTVTBK2E7IGYOYZMK6TTHKMMBST52RCZVKM776PV3USLXONMH66Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.349296225 (5h14m17.698603501s) [noise=true , address=89.149.201.147:4011 , piece_id=KQM7L432OU7BYKWL2QHAIKRNPKPA7NOPSZJDOAAFQEGV66PZMGTA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.352514193 (5h14m17.701821479s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.380253194 (5h14m17.72956049s) [noise=true , address=185.167.204.205:4030 , piece_id=35Y5DRDZZHA7LEUZDJQ36RU4MB6DQN2AAIG7E45EIC6YM3MQ7HHQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.385432055 (5h14m17.734739341s) [noise=true , address=185.167.204.205:4023 , piece_id=ACMLGJMZ5H25VH7VULKAIBWIHIAEE2AV366UEZFOJDZRNQ4NGYLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.388082382 (5h14m17.737389678s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.391016018 (5h14m17.740323304s) [noise=true , address=185.167.204.206:4056 , piece_id=G5D6SI6CVQHAVSAPS6GJRVZIQ4YQ23PFH7KL26TVK4I4BBTW2C5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.396014777 (5h14m17.745322053s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.401559615 (5h14m17.750866901s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.414355180 (5h14m17.763662466s) [noise=true , address=185.167.204.206:4041 , piece_id=NUWDUCCNI44ANZLNYISQQKFSBK7WYTPAPGQ6JJ43IP44LNCLT4SA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.421539241 (5h14m17.770846537s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.423334288 (5h14m17.772641574s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.447102799 (5h14m17.796410075s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.473754258 (5h14m17.823061544s) [noise=true , address=185.167.204.206:4066 , piece_id=KKDV46RFAENAUYDYRPDCV64LWX5RB7JVRKB3HY7BXB4LU7DITSLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.481151965 (5h14m17.830459251s) [noise=true , address=185.167.204.205:4019 , piece_id=7LYDTS6D3HQSDMVBCMRLB5NZIRLP44VRSWCUT5EKSGSRPWP4RHIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.486154732 (5h14m17.835462018s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:57.813202781 (5h15m15.162510068s) [noise=true , address=185.167.204.206:4053 , piece_id=UFA7HABPN4LRP7RSCHBHNXT5AMEDFUF3DSYEGWBFSXOTOXRQXT7A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: rpc: dial tcp 185.167.204.206:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813463338 (5h15m15.162770614s) [noise=true , address=185.167.204.206:4047 , piece_id=Z2QX5FHJKEZVW4FXDV2NNUMXEBVPGEW4Z7C3GBVZRUTNFR4MA5IA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: write tcp 188.166.225.170:59506->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813491391 (5h15m15.162798677s) [noise=true , address=185.167.204.205:4013 , piece_id=KICB6KLFDM3KFOGJ52UVXO3G4BUPSSJF657ZIWUIVP56M2SLGMWA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: context deadline exceeded; context deadline exceeded Oct 27 19:57:57.813536527 (5h15m15.162843813s) [seg_pos={0 0}, upload=4621]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Oct 27 19:57:58.016780515 (5h15m15.366087802s) [upload=4621]: Done uploading segments... Oct 27 19:57:58.016879354 (5h15m15.36618663s) [upload=4621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.095802609s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=3) [.XX..X......................................................] duration: 3.361343532s FAIL (2025-10-27 20:11:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.361371674s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.047974055s FAIL (2025-10-27 20:11:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== eg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577374904 (5h14m15.926682211s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577388009 (5h14m15.926695285s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577699713 (5h14m15.927006989s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577834229 (5h14m15.927141505s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578014502 (5h14m15.927321778s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578280880 (5h14m15.927588156s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578608904 (5h14m15.92791618s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578935426 (5h14m15.928242712s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579041648 (5h14m15.928348924s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579222723 (5h14m15.928529999s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579486625 (5h14m15.928793901s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579780545 (5h14m15.929087821s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.580102097 (5h14m15.929409403s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:59.402940250 (5h14m16.752247547s) [upload=4621]: Next returned nil segment Oct 27 19:56:59.402970929 (5h14m16.752278235s) [upload=4621]: Waiting for error group managing segments... Oct 27 19:56:59.801926071 (5h14m17.151233347s) [noise=true , address=81.171.14.70:4034 , piece_id=QQXBWVXWX24ETMCIFZ7LTONF3LVGL3RLDXXSIQGX62PLWPJSFM5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.811249835 (5h14m17.160557121s) [noise=true , address=81.171.14.70:4048 , piece_id=PR2HDQTP5VKTMF7HG6VNKKUJD66LC2HKQVAZUGEZKKZNKH2QCZFQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.858197924 (5h14m17.20750521s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.897902089 (5h14m17.247209375s) [noise=true , address=89.149.201.147:4023 , piece_id=LEEWZDLYWPSSUDRIPEQK4NKC4PFF6JVLEBPZMQJLWCG4YZHRFQ3A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.901773520 (5h14m17.251080806s) [noise=true , address=89.149.201.147:4010 , piece_id=TRY6HG262YJRDUVNLLHRYG554BVP4J5TWPGHJUVN5N55WHU3C4ZA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902238556 (5h14m17.251545842s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902438327 (5h14m17.251745603s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.917540715 (5h14m17.266847991s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.926674337 (5h14m17.275981623s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.328819374 (5h14m17.67812666s) [noise=true , address=81.171.14.70:4030 , piece_id=ZTVTBK2E7IGYOYZMK6TTHKMMBST52RCZVKM776PV3USLXONMH66Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.349296225 (5h14m17.698603501s) [noise=true , address=89.149.201.147:4011 , piece_id=KQM7L432OU7BYKWL2QHAIKRNPKPA7NOPSZJDOAAFQEGV66PZMGTA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.352514193 (5h14m17.701821479s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.380253194 (5h14m17.72956049s) [noise=true , address=185.167.204.205:4030 , piece_id=35Y5DRDZZHA7LEUZDJQ36RU4MB6DQN2AAIG7E45EIC6YM3MQ7HHQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.385432055 (5h14m17.734739341s) [noise=true , address=185.167.204.205:4023 , piece_id=ACMLGJMZ5H25VH7VULKAIBWIHIAEE2AV366UEZFOJDZRNQ4NGYLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.388082382 (5h14m17.737389678s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.391016018 (5h14m17.740323304s) [noise=true , address=185.167.204.206:4056 , piece_id=G5D6SI6CVQHAVSAPS6GJRVZIQ4YQ23PFH7KL26TVK4I4BBTW2C5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.396014777 (5h14m17.745322053s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.401559615 (5h14m17.750866901s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.414355180 (5h14m17.763662466s) [noise=true , address=185.167.204.206:4041 , piece_id=NUWDUCCNI44ANZLNYISQQKFSBK7WYTPAPGQ6JJ43IP44LNCLT4SA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.421539241 (5h14m17.770846537s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.423334288 (5h14m17.772641574s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.447102799 (5h14m17.796410075s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.473754258 (5h14m17.823061544s) [noise=true , address=185.167.204.206:4066 , piece_id=KKDV46RFAENAUYDYRPDCV64LWX5RB7JVRKB3HY7BXB4LU7DITSLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.481151965 (5h14m17.830459251s) [noise=true , address=185.167.204.205:4019 , piece_id=7LYDTS6D3HQSDMVBCMRLB5NZIRLP44VRSWCUT5EKSGSRPWP4RHIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.486154732 (5h14m17.835462018s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:57.813202781 (5h15m15.162510068s) [noise=true , address=185.167.204.206:4053 , piece_id=UFA7HABPN4LRP7RSCHBHNXT5AMEDFUF3DSYEGWBFSXOTOXRQXT7A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: rpc: dial tcp 185.167.204.206:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813463338 (5h15m15.162770614s) [noise=true , address=185.167.204.206:4047 , piece_id=Z2QX5FHJKEZVW4FXDV2NNUMXEBVPGEW4Z7C3GBVZRUTNFR4MA5IA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: write tcp 188.166.225.170:59506->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813491391 (5h15m15.162798677s) [noise=true , address=185.167.204.205:4013 , piece_id=KICB6KLFDM3KFOGJ52UVXO3G4BUPSSJF657ZIWUIVP56M2SLGMWA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: context deadline exceeded; context deadline exceeded Oct 27 19:57:57.813536527 (5h15m15.162843813s) [seg_pos={0 0}, upload=4621]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Oct 27 19:57:58.016780515 (5h15m15.366087802s) [upload=4621]: Done uploading segments... Oct 27 19:57:58.016879354 (5h15m15.36618663s) [upload=4621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.048004372s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.117915926s FAIL (2025-10-27 20:10:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== eg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577374904 (5h14m15.926682211s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577388009 (5h14m15.926695285s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577699713 (5h14m15.927006989s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577834229 (5h14m15.927141505s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578014502 (5h14m15.927321778s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578280880 (5h14m15.927588156s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578608904 (5h14m15.92791618s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578935426 (5h14m15.928242712s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579041648 (5h14m15.928348924s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579222723 (5h14m15.928529999s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579486625 (5h14m15.928793901s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579780545 (5h14m15.929087821s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.580102097 (5h14m15.929409403s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:59.402940250 (5h14m16.752247547s) [upload=4621]: Next returned nil segment Oct 27 19:56:59.402970929 (5h14m16.752278235s) [upload=4621]: Waiting for error group managing segments... Oct 27 19:56:59.801926071 (5h14m17.151233347s) [noise=true , address=81.171.14.70:4034 , piece_id=QQXBWVXWX24ETMCIFZ7LTONF3LVGL3RLDXXSIQGX62PLWPJSFM5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.811249835 (5h14m17.160557121s) [noise=true , address=81.171.14.70:4048 , piece_id=PR2HDQTP5VKTMF7HG6VNKKUJD66LC2HKQVAZUGEZKKZNKH2QCZFQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.858197924 (5h14m17.20750521s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.897902089 (5h14m17.247209375s) [noise=true , address=89.149.201.147:4023 , piece_id=LEEWZDLYWPSSUDRIPEQK4NKC4PFF6JVLEBPZMQJLWCG4YZHRFQ3A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.901773520 (5h14m17.251080806s) [noise=true , address=89.149.201.147:4010 , piece_id=TRY6HG262YJRDUVNLLHRYG554BVP4J5TWPGHJUVN5N55WHU3C4ZA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902238556 (5h14m17.251545842s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902438327 (5h14m17.251745603s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.917540715 (5h14m17.266847991s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.926674337 (5h14m17.275981623s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.328819374 (5h14m17.67812666s) [noise=true , address=81.171.14.70:4030 , piece_id=ZTVTBK2E7IGYOYZMK6TTHKMMBST52RCZVKM776PV3USLXONMH66Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.349296225 (5h14m17.698603501s) [noise=true , address=89.149.201.147:4011 , piece_id=KQM7L432OU7BYKWL2QHAIKRNPKPA7NOPSZJDOAAFQEGV66PZMGTA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.352514193 (5h14m17.701821479s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.380253194 (5h14m17.72956049s) [noise=true , address=185.167.204.205:4030 , piece_id=35Y5DRDZZHA7LEUZDJQ36RU4MB6DQN2AAIG7E45EIC6YM3MQ7HHQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.385432055 (5h14m17.734739341s) [noise=true , address=185.167.204.205:4023 , piece_id=ACMLGJMZ5H25VH7VULKAIBWIHIAEE2AV366UEZFOJDZRNQ4NGYLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.388082382 (5h14m17.737389678s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.391016018 (5h14m17.740323304s) [noise=true , address=185.167.204.206:4056 , piece_id=G5D6SI6CVQHAVSAPS6GJRVZIQ4YQ23PFH7KL26TVK4I4BBTW2C5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.396014777 (5h14m17.745322053s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.401559615 (5h14m17.750866901s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.414355180 (5h14m17.763662466s) [noise=true , address=185.167.204.206:4041 , piece_id=NUWDUCCNI44ANZLNYISQQKFSBK7WYTPAPGQ6JJ43IP44LNCLT4SA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.421539241 (5h14m17.770846537s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.423334288 (5h14m17.772641574s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.447102799 (5h14m17.796410075s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.473754258 (5h14m17.823061544s) [noise=true , address=185.167.204.206:4066 , piece_id=KKDV46RFAENAUYDYRPDCV64LWX5RB7JVRKB3HY7BXB4LU7DITSLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.481151965 (5h14m17.830459251s) [noise=true , address=185.167.204.205:4019 , piece_id=7LYDTS6D3HQSDMVBCMRLB5NZIRLP44VRSWCUT5EKSGSRPWP4RHIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.486154732 (5h14m17.835462018s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:57.813202781 (5h15m15.162510068s) [noise=true , address=185.167.204.206:4053 , piece_id=UFA7HABPN4LRP7RSCHBHNXT5AMEDFUF3DSYEGWBFSXOTOXRQXT7A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: rpc: dial tcp 185.167.204.206:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813463338 (5h15m15.162770614s) [noise=true , address=185.167.204.206:4047 , piece_id=Z2QX5FHJKEZVW4FXDV2NNUMXEBVPGEW4Z7C3GBVZRUTNFR4MA5IA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: write tcp 188.166.225.170:59506->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813491391 (5h15m15.162798677s) [noise=true , address=185.167.204.205:4013 , piece_id=KICB6KLFDM3KFOGJ52UVXO3G4BUPSSJF657ZIWUIVP56M2SLGMWA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: context deadline exceeded; context deadline exceeded Oct 27 19:57:57.813536527 (5h15m15.162843813s) [seg_pos={0 0}, upload=4621]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Oct 27 19:57:58.016780515 (5h15m15.366087802s) [upload=4621]: Done uploading segments... Oct 27 19:57:58.016879354 (5h15m15.36618663s) [upload=4621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.117950954s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=3) [..XX..X.....................................................] duration: 4.199269771s FAIL (2025-10-27 20:10:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.19929508s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 4.941965644s FAIL (2025-10-27 20:09:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== eg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577374904 (5h14m15.926682211s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577388009 (5h14m15.926695285s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577699713 (5h14m15.927006989s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577834229 (5h14m15.927141505s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578014502 (5h14m15.927321778s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578280880 (5h14m15.927588156s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578608904 (5h14m15.92791618s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578935426 (5h14m15.928242712s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579041648 (5h14m15.928348924s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579222723 (5h14m15.928529999s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579486625 (5h14m15.928793901s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579780545 (5h14m15.929087821s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.580102097 (5h14m15.929409403s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:59.402940250 (5h14m16.752247547s) [upload=4621]: Next returned nil segment Oct 27 19:56:59.402970929 (5h14m16.752278235s) [upload=4621]: Waiting for error group managing segments... Oct 27 19:56:59.801926071 (5h14m17.151233347s) [noise=true , address=81.171.14.70:4034 , piece_id=QQXBWVXWX24ETMCIFZ7LTONF3LVGL3RLDXXSIQGX62PLWPJSFM5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.811249835 (5h14m17.160557121s) [noise=true , address=81.171.14.70:4048 , piece_id=PR2HDQTP5VKTMF7HG6VNKKUJD66LC2HKQVAZUGEZKKZNKH2QCZFQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.858197924 (5h14m17.20750521s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.897902089 (5h14m17.247209375s) [noise=true , address=89.149.201.147:4023 , piece_id=LEEWZDLYWPSSUDRIPEQK4NKC4PFF6JVLEBPZMQJLWCG4YZHRFQ3A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.901773520 (5h14m17.251080806s) [noise=true , address=89.149.201.147:4010 , piece_id=TRY6HG262YJRDUVNLLHRYG554BVP4J5TWPGHJUVN5N55WHU3C4ZA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902238556 (5h14m17.251545842s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902438327 (5h14m17.251745603s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.917540715 (5h14m17.266847991s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.926674337 (5h14m17.275981623s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.328819374 (5h14m17.67812666s) [noise=true , address=81.171.14.70:4030 , piece_id=ZTVTBK2E7IGYOYZMK6TTHKMMBST52RCZVKM776PV3USLXONMH66Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.349296225 (5h14m17.698603501s) [noise=true , address=89.149.201.147:4011 , piece_id=KQM7L432OU7BYKWL2QHAIKRNPKPA7NOPSZJDOAAFQEGV66PZMGTA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.352514193 (5h14m17.701821479s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.380253194 (5h14m17.72956049s) [noise=true , address=185.167.204.205:4030 , piece_id=35Y5DRDZZHA7LEUZDJQ36RU4MB6DQN2AAIG7E45EIC6YM3MQ7HHQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.385432055 (5h14m17.734739341s) [noise=true , address=185.167.204.205:4023 , piece_id=ACMLGJMZ5H25VH7VULKAIBWIHIAEE2AV366UEZFOJDZRNQ4NGYLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.388082382 (5h14m17.737389678s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.391016018 (5h14m17.740323304s) [noise=true , address=185.167.204.206:4056 , piece_id=G5D6SI6CVQHAVSAPS6GJRVZIQ4YQ23PFH7KL26TVK4I4BBTW2C5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.396014777 (5h14m17.745322053s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.401559615 (5h14m17.750866901s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.414355180 (5h14m17.763662466s) [noise=true , address=185.167.204.206:4041 , piece_id=NUWDUCCNI44ANZLNYISQQKFSBK7WYTPAPGQ6JJ43IP44LNCLT4SA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.421539241 (5h14m17.770846537s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.423334288 (5h14m17.772641574s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.447102799 (5h14m17.796410075s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.473754258 (5h14m17.823061544s) [noise=true , address=185.167.204.206:4066 , piece_id=KKDV46RFAENAUYDYRPDCV64LWX5RB7JVRKB3HY7BXB4LU7DITSLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.481151965 (5h14m17.830459251s) [noise=true , address=185.167.204.205:4019 , piece_id=7LYDTS6D3HQSDMVBCMRLB5NZIRLP44VRSWCUT5EKSGSRPWP4RHIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.486154732 (5h14m17.835462018s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:57.813202781 (5h15m15.162510068s) [noise=true , address=185.167.204.206:4053 , piece_id=UFA7HABPN4LRP7RSCHBHNXT5AMEDFUF3DSYEGWBFSXOTOXRQXT7A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: rpc: dial tcp 185.167.204.206:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813463338 (5h15m15.162770614s) [noise=true , address=185.167.204.206:4047 , piece_id=Z2QX5FHJKEZVW4FXDV2NNUMXEBVPGEW4Z7C3GBVZRUTNFR4MA5IA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: write tcp 188.166.225.170:59506->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813491391 (5h15m15.162798677s) [noise=true , address=185.167.204.205:4013 , piece_id=KICB6KLFDM3KFOGJ52UVXO3G4BUPSSJF657ZIWUIVP56M2SLGMWA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: context deadline exceeded; context deadline exceeded Oct 27 19:57:57.813536527 (5h15m15.162843813s) [seg_pos={0 0}, upload=4621]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Oct 27 19:57:58.016780515 (5h15m15.366087802s) [upload=4621]: Done uploading segments... Oct 27 19:57:58.016879354 (5h15m15.36618663s) [upload=4621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 4.941997214s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=3) [...XX..X....................................................] duration: 3.41752911s FAIL (2025-10-27 20:09:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.417594214s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.004356567s FAIL (2025-10-27 20:08:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== eg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577374904 (5h14m15.926682211s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577388009 (5h14m15.926695285s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577699713 (5h14m15.927006989s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577834229 (5h14m15.927141505s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578014502 (5h14m15.927321778s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578280880 (5h14m15.927588156s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578608904 (5h14m15.92791618s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578935426 (5h14m15.928242712s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579041648 (5h14m15.928348924s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579222723 (5h14m15.928529999s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579486625 (5h14m15.928793901s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579780545 (5h14m15.929087821s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.580102097 (5h14m15.929409403s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:59.402940250 (5h14m16.752247547s) [upload=4621]: Next returned nil segment Oct 27 19:56:59.402970929 (5h14m16.752278235s) [upload=4621]: Waiting for error group managing segments... Oct 27 19:56:59.801926071 (5h14m17.151233347s) [noise=true , address=81.171.14.70:4034 , piece_id=QQXBWVXWX24ETMCIFZ7LTONF3LVGL3RLDXXSIQGX62PLWPJSFM5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.811249835 (5h14m17.160557121s) [noise=true , address=81.171.14.70:4048 , piece_id=PR2HDQTP5VKTMF7HG6VNKKUJD66LC2HKQVAZUGEZKKZNKH2QCZFQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.858197924 (5h14m17.20750521s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.897902089 (5h14m17.247209375s) [noise=true , address=89.149.201.147:4023 , piece_id=LEEWZDLYWPSSUDRIPEQK4NKC4PFF6JVLEBPZMQJLWCG4YZHRFQ3A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.901773520 (5h14m17.251080806s) [noise=true , address=89.149.201.147:4010 , piece_id=TRY6HG262YJRDUVNLLHRYG554BVP4J5TWPGHJUVN5N55WHU3C4ZA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902238556 (5h14m17.251545842s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902438327 (5h14m17.251745603s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.917540715 (5h14m17.266847991s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.926674337 (5h14m17.275981623s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.328819374 (5h14m17.67812666s) [noise=true , address=81.171.14.70:4030 , piece_id=ZTVTBK2E7IGYOYZMK6TTHKMMBST52RCZVKM776PV3USLXONMH66Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.349296225 (5h14m17.698603501s) [noise=true , address=89.149.201.147:4011 , piece_id=KQM7L432OU7BYKWL2QHAIKRNPKPA7NOPSZJDOAAFQEGV66PZMGTA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.352514193 (5h14m17.701821479s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.380253194 (5h14m17.72956049s) [noise=true , address=185.167.204.205:4030 , piece_id=35Y5DRDZZHA7LEUZDJQ36RU4MB6DQN2AAIG7E45EIC6YM3MQ7HHQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.385432055 (5h14m17.734739341s) [noise=true , address=185.167.204.205:4023 , piece_id=ACMLGJMZ5H25VH7VULKAIBWIHIAEE2AV366UEZFOJDZRNQ4NGYLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.388082382 (5h14m17.737389678s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.391016018 (5h14m17.740323304s) [noise=true , address=185.167.204.206:4056 , piece_id=G5D6SI6CVQHAVSAPS6GJRVZIQ4YQ23PFH7KL26TVK4I4BBTW2C5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.396014777 (5h14m17.745322053s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.401559615 (5h14m17.750866901s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.414355180 (5h14m17.763662466s) [noise=true , address=185.167.204.206:4041 , piece_id=NUWDUCCNI44ANZLNYISQQKFSBK7WYTPAPGQ6JJ43IP44LNCLT4SA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.421539241 (5h14m17.770846537s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.423334288 (5h14m17.772641574s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.447102799 (5h14m17.796410075s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.473754258 (5h14m17.823061544s) [noise=true , address=185.167.204.206:4066 , piece_id=KKDV46RFAENAUYDYRPDCV64LWX5RB7JVRKB3HY7BXB4LU7DITSLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.481151965 (5h14m17.830459251s) [noise=true , address=185.167.204.205:4019 , piece_id=7LYDTS6D3HQSDMVBCMRLB5NZIRLP44VRSWCUT5EKSGSRPWP4RHIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.486154732 (5h14m17.835462018s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:57.813202781 (5h15m15.162510068s) [noise=true , address=185.167.204.206:4053 , piece_id=UFA7HABPN4LRP7RSCHBHNXT5AMEDFUF3DSYEGWBFSXOTOXRQXT7A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: rpc: dial tcp 185.167.204.206:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813463338 (5h15m15.162770614s) [noise=true , address=185.167.204.206:4047 , piece_id=Z2QX5FHJKEZVW4FXDV2NNUMXEBVPGEW4Z7C3GBVZRUTNFR4MA5IA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: write tcp 188.166.225.170:59506->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813491391 (5h15m15.162798677s) [noise=true , address=185.167.204.205:4013 , piece_id=KICB6KLFDM3KFOGJ52UVXO3G4BUPSSJF657ZIWUIVP56M2SLGMWA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: context deadline exceeded; context deadline exceeded Oct 27 19:57:57.813536527 (5h15m15.162843813s) [seg_pos={0 0}, upload=4621]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Oct 27 19:57:58.016780515 (5h15m15.366087802s) [upload=4621]: Done uploading segments... Oct 27 19:57:58.016879354 (5h15m15.36618663s) [upload=4621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.004386723s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=3) [....XX..X...................................................] duration: 3.419333774s FAIL (2025-10-27 20:08:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.41935249s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 4.973108774s FAIL (2025-10-27 20:07:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== eg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577374904 (5h14m15.926682211s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577388009 (5h14m15.926695285s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577699713 (5h14m15.927006989s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.577834229 (5h14m15.927141505s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578014502 (5h14m15.927321778s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578280880 (5h14m15.927588156s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578608904 (5h14m15.92791618s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.578935426 (5h14m15.928242712s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579041648 (5h14m15.928348924s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579222723 (5h14m15.928529999s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579486625 (5h14m15.928793901s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.579780545 (5h14m15.929087821s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:58.580102097 (5h14m15.929409403s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Uploading piece... Oct 27 19:56:59.402940250 (5h14m16.752247547s) [upload=4621]: Next returned nil segment Oct 27 19:56:59.402970929 (5h14m16.752278235s) [upload=4621]: Waiting for error group managing segments... Oct 27 19:56:59.801926071 (5h14m17.151233347s) [noise=true , address=81.171.14.70:4034 , piece_id=QQXBWVXWX24ETMCIFZ7LTONF3LVGL3RLDXXSIQGX62PLWPJSFM5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.811249835 (5h14m17.160557121s) [noise=true , address=81.171.14.70:4048 , piece_id=PR2HDQTP5VKTMF7HG6VNKKUJD66LC2HKQVAZUGEZKKZNKH2QCZFQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.858197924 (5h14m17.20750521s) [noise=true , address=89.149.201.147:4007 , piece_id=6YAZLKZQ3IRZ3KU4M2MVAAOBEPJZGLFJAS6A4BPSNUPCNNVQHDDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.897902089 (5h14m17.247209375s) [noise=true , address=89.149.201.147:4023 , piece_id=LEEWZDLYWPSSUDRIPEQK4NKC4PFF6JVLEBPZMQJLWCG4YZHRFQ3A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.901773520 (5h14m17.251080806s) [noise=true , address=89.149.201.147:4010 , piece_id=TRY6HG262YJRDUVNLLHRYG554BVP4J5TWPGHJUVN5N55WHU3C4ZA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902238556 (5h14m17.251545842s) [noise=true , address=89.149.201.147:4005 , piece_id=27IBEU4BYZCGDNOUDCRTQZFZ35W3BLQGERULMCKVWFGWZ7676RPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.902438327 (5h14m17.251745603s) [noise=true , address=81.171.14.70:4039 , piece_id=S5634HCFPYT7SJGH5BPZ3OJZDIAJ5YYSXCUWGIYVIJ2WPQAEWEPQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.917540715 (5h14m17.266847991s) [noise=true , address=89.149.201.147:4017 , piece_id=Q4AHXE26ASEWAANQIW55O433TDNA3RPYX7KCEZVPDET264I5DV3Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:56:59.926674337 (5h14m17.275981623s) [noise=true , address=89.149.201.147:4020 , piece_id=7O5NTDXJMZL7XX7EJJZMSCLQPS2KBXW2ASVMGFKBKDASUOJ5KYQA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.328819374 (5h14m17.67812666s) [noise=true , address=81.171.14.70:4030 , piece_id=ZTVTBK2E7IGYOYZMK6TTHKMMBST52RCZVKM776PV3USLXONMH66Q, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.349296225 (5h14m17.698603501s) [noise=true , address=89.149.201.147:4011 , piece_id=KQM7L432OU7BYKWL2QHAIKRNPKPA7NOPSZJDOAAFQEGV66PZMGTA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.352514193 (5h14m17.701821479s) [noise=true , address=89.149.201.147:4012 , piece_id=VGEAVYUXQB6F4OF7BCCCZGGI4S2OT5BXXDFSEMHGLMOUAPFCPXDQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.380253194 (5h14m17.72956049s) [noise=true , address=185.167.204.205:4030 , piece_id=35Y5DRDZZHA7LEUZDJQ36RU4MB6DQN2AAIG7E45EIC6YM3MQ7HHQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.385432055 (5h14m17.734739341s) [noise=true , address=185.167.204.205:4023 , piece_id=ACMLGJMZ5H25VH7VULKAIBWIHIAEE2AV366UEZFOJDZRNQ4NGYLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.388082382 (5h14m17.737389678s) [noise=true , address=185.167.204.206:4040 , piece_id=PDSOHIACMQCDA6NQQZWS4YUI57BMU4DJY5GKRPR7JVLLFVZXDJXQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.391016018 (5h14m17.740323304s) [noise=true , address=185.167.204.206:4056 , piece_id=G5D6SI6CVQHAVSAPS6GJRVZIQ4YQ23PFH7KL26TVK4I4BBTW2C5A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.396014777 (5h14m17.745322053s) [noise=true , address=185.167.204.206:4052 , piece_id=RTBLKXBTFLJOSBGSMSZ6HJNPDSM5GPO67VVRHP3OAK5DF5QFAVCQ, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.401559615 (5h14m17.750866901s) [noise=true , address=185.167.204.205:4002 , piece_id=U57ZVYRVT2L35FNA2N6ODTLEP3PN3TPP5ZYSONBXJYXWE5K43S6A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.414355180 (5h14m17.763662466s) [noise=true , address=185.167.204.206:4041 , piece_id=NUWDUCCNI44ANZLNYISQQKFSBK7WYTPAPGQ6JJ43IP44LNCLT4SA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.421539241 (5h14m17.770846537s) [noise=true , address=185.167.204.206:4048 , piece_id=HD65TI6UG22CW34RKIVC5RMYIOUIYHL72BJZYL55XHCX3HQMZYHA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.423334288 (5h14m17.772641574s) [noise=true , address=185.167.204.205:4010 , piece_id=Q35AS6W5CQL2OMHIEPI7GVR6NS7IEJ6LCFXZEWGFEWL6XENNNVIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.447102799 (5h14m17.796410075s) [noise=true , address=185.167.204.205:4012 , piece_id=JKUFH3UD6ACMEQWRFYJ2F5P5ZJQKYCA6E7FXTHBF7MNQA6EKLXGA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.473754258 (5h14m17.823061544s) [noise=true , address=185.167.204.206:4066 , piece_id=KKDV46RFAENAUYDYRPDCV64LWX5RB7JVRKB3HY7BXB4LU7DITSLA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.481151965 (5h14m17.830459251s) [noise=true , address=185.167.204.205:4019 , piece_id=7LYDTS6D3HQSDMVBCMRLB5NZIRLP44VRSWCUT5EKSGSRPWP4RHIA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:00.486154732 (5h14m17.835462018s) [noise=true , address=185.167.204.205:4001 , piece_id=Z22S36XDBYL5COSP36N575LMSLJVC6AP4ZXFNXNMRTVN23HTP5XA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: <nil> Oct 27 19:57:57.813202781 (5h15m15.162510068s) [noise=true , address=185.167.204.206:4053 , piece_id=UFA7HABPN4LRP7RSCHBHNXT5AMEDFUF3DSYEGWBFSXOTOXRQXT7A, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: rpc: dial tcp 185.167.204.206:4053: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813463338 (5h15m15.162770614s) [noise=true , address=185.167.204.206:4047 , piece_id=Z2QX5FHJKEZVW4FXDV2NNUMXEBVPGEW4Z7C3GBVZRUTNFR4MA5IA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:12M7W4Yno5rMgemSC7nJd42asmZXqUY9cCyY7mtFFHEh8iqAhCF, address:185.167.204.206:4047): protocol: write tcp 188.166.225.170:59506->185.167.204.206:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 27 19:57:57.813491391 (5h15m15.162798677s) [noise=true , address=185.167.204.205:4013 , piece_id=KICB6KLFDM3KFOGJ52UVXO3G4BUPSSJF657ZIWUIVP56M2SLGMWA, seg_pos={0 0}, upload=4621]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: context deadline exceeded; context deadline exceeded Oct 27 19:57:57.813536527 (5h15m15.162843813s) [seg_pos={0 0}, upload=4621]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Oct 27 19:57:58.016780515 (5h15m15.366087802s) [upload=4621]: Done uploading segments... Oct 27 19:57:58.016879354 (5h15m15.36618663s) [upload=4621]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 4.973160253s core uplink eu-select-1 eu1-placement-15

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)