Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [............................X....X..X.X.....................] duration: 9.80451788s FAIL (2025-09-27 19:43:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 27 19:27:22.886943307 (101h56m8.34903578s) [upload=89705]: Starting upload Sep 27 19:27:22.887049439 (101h56m8.349141922s) [upload=89705]: Done starting upload Sep 27 19:27:22.887160700 (101h56m8.349253184s) [upload=89705]: Uploading segments... Sep 27 19:27:22.887344050 (101h56m8.349436533s) [upload=89705]: Got next segment. Inline: false Sep 27 19:27:22.887393123 (101h56m8.349485606s) [upload=89705]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:27:22.887400948 (101h56m8.349493431s) [upload=89705]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:27:23.144152141 (101h56m8.606244624s) [seg_pos={0 0}, upload=89705]: Begin upload segment... Sep 27 19:27:23.144224288 (101h56m8.606316771s) [seg_pos={0 0}, upload=89705]: Done begin upload segment. Sep 27 19:27:23.144303519 (101h56m8.606396002s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144356600 (101h56m8.606449083s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144899694 (101h56m8.606992177s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145386381 (101h56m8.607478864s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145749413 (101h56m8.607841896s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146190593 (101h56m8.608283076s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146569114 (101h56m8.608661597s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146975998 (101h56m8.609068481s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.147437558 (101h56m8.609530051s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.148991526 (101h56m8.611083999s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.151370226 (101h56m8.613462699s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.153185462 (101h56m8.615277966s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.154735545 (101h56m8.616828028s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.158605123 (101h56m8.620697606s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162754133 (101h56m8.624846616s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162908967 (101h56m8.62500145s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:24.838034007 (101h56m10.30012649s) [upload=89705]: Next returned nil segment Sep 27 19:27:24.838098610 (101h56m10.300191093s) [upload=89705]: Waiting for error group managing segments... Sep 27 19:27:25.171482756 (101h56m10.633575229s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.176947972 (101h56m10.639040456s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.180229350 (101h56m10.642321834s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.181707686 (101h56m10.643800159s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.182762434 (101h56m10.644854917s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.192828732 (101h56m10.654921205s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.198888852 (101h56m10.660981335s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.205614519 (101h56m10.667707012s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.207989521 (101h56m10.670081994s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.222032181 (101h56m10.684124654s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.290677625 (101h56m10.752770118s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.652827506 (101h56m11.114919989s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:52.180260159 (101h56m37.642352652s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180423099 (101h56m37.642515582s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180455370 (101h56m37.642547853s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180565359 (101h56m37.642657842s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:37784->77.90.31.15:4697: use of closed network connection; conn closed: op Sep 27 19:27:52.180610105 (101h56m37.642702588s) [seg_pos={0 0}, upload=89705]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 27 19:27:52.433487771 (101h56m37.895580254s) [upload=89705]: Done uploading segments... Sep 27 19:27:52.433517006 (101h56m37.89560948s) [upload=89705]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 9.804551694s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 4.882249196s FAIL (2025-09-27 19:43:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: manager closed: closed: read tcp 188.166.225.170:34184->35.207.121.91:7777: read: connection reset by peer ======LOGS====== ploading piece... Sep 27 19:42:16.851904981 (102h11m2.313997454s) [noise=true , address=185.167.204.205:4019 , piece_id=7IDC4PR2J5BSCO7LDYL4B5ZH72LERJ53QQ3CJQVEUSZBABQJFMRQ, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.852399574 (102h11m2.314492047s) [noise=true , address=185.167.204.206:4069 , piece_id=KGO3J74KNEVFROVWK4BP7FUE6C7HNGRPWNFUCRQQBQE5ZGQ4F27Q, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.852873816 (102h11m2.314966299s) [noise=true , address=185.167.204.205:4016 , piece_id=GYH5PXP335XI5C6BEZZ6IIFXI7CGHTRMX5JSGEOB25NRTEEUP46A, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.853507803 (102h11m2.315600276s) [noise=true , address=185.167.204.206:4040 , piece_id=IKSYLPGG2N2WNMYMJ2GE6LZOVS2ZLIWQDMO353TEOD52YDLQNFGQ, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.855290378 (102h11m2.317382851s) [noise=true , address=81.171.14.70:4039 , piece_id=DI7DICLIQLQBNTR62K6KT2Z3F4PJMPEGRDAJJFHQNUF4D53WHQNQ, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.857320493 (102h11m2.319412967s) [noise=true , address=81.171.14.70:4028 , piece_id=TKDL5V3DXRB6IEX4UCDIORWH5C2JEA2SJV4FURSB2E43TUR5BVOQ, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.859477411 (102h11m2.321569884s) [noise=true , address=81.171.14.70:4036 , piece_id=WAGPANGKB4NLD75UKGEQM2V53LSKLZCAMRIXNAYMPKXQYD5ZJSHA, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.861648736 (102h11m2.323741219s) [noise=true , address=89.149.201.147:4001 , piece_id=S4GZ34GDBSFVRS52KNUFSZFTPD4TDTFUMHUKWZSMNAHSFLZHZ5QQ, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.863879203 (102h11m2.325971686s) [noise=true , address=185.167.204.206:4051 , piece_id=62564CXV5ZNICNGEZMA4ZV4CZYHJW4NJZU64ARYG772SQ3MD5A2Q, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.866136491 (102h11m2.328228974s) [noise=true , address=81.171.14.70:4045 , piece_id=HESWYJWN53AC7JEYBA2LZJPE5N4E72MMVSAK4OOYRSUFKLPGS45Q, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:17.717331390 (102h11m3.179423873s) [upload=89929]: Next returned nil segment Sep 27 19:42:17.717367178 (102h11m3.179459662s) [upload=89929]: Waiting for error group managing segments... Sep 27 19:42:18.087365918 (102h11m3.549458401s) [noise=true , address=81.171.14.70:4035 , piece_id=M3OI7SKEGYRTQCLXDWE364IABTZEDGP6QXWSQHHYGXTPP3VCLDZA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.105655594 (102h11m3.567748077s) [noise=true , address=81.171.14.70:4041 , piece_id=HESUR7NWPV427S2XRODHYHLS5KLNQLPOTPF3HOVNDQNRTUWGZ7KA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.107753430 (102h11m3.569845913s) [noise=true , address=81.171.14.70:4037 , piece_id=NF4KCWJHMWPWVF2SZLIHLFIJDH6TNCTDSQT3EQDBYMU6HHYBJ5OA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.114636617 (102h11m3.5767291s) [noise=true , address=81.171.14.70:4031 , piece_id=332OHLFNBAJ7LB36Y72MNALDU7NPXQIRY5QRVLAPJOTE6TCDOCEA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.125611034 (102h11m3.587703527s) [noise=true , address=81.171.14.70:4028 , piece_id=TKDL5V3DXRB6IEX4UCDIORWH5C2JEA2SJV4FURSB2E43TUR5BVOQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.130988554 (102h11m3.593081037s) [noise=true , address=81.171.14.70:4039 , piece_id=DI7DICLIQLQBNTR62K6KT2Z3F4PJMPEGRDAJJFHQNUF4D53WHQNQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.133905207 (102h11m3.59599769s) [noise=true , address=81.171.14.70:4047 , piece_id=2SEMSVDNNEQGCTKTJ2NPIXUAGFIHQ7B32O2FEOSFLYC55ZFWAE5Q, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.137235840 (102h11m3.599328313s) [noise=true , address=89.149.201.147:4019 , piece_id=DQJBIPFQLRH7PVC5ET7YSKJNQ4ZXT2BR6LHX74ZHRVHLIHEQXXUQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.138020274 (102h11m3.600112747s) [noise=true , address=89.149.201.147:4009 , piece_id=ZFTEYLRPQPK3MPAQS5OPXGZHJMAFOACLFRROWVIPXEZQFUP2ZGVQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.169014162 (102h11m3.631106645s) [noise=true , address=81.171.14.70:4036 , piece_id=WAGPANGKB4NLD75UKGEQM2V53LSKLZCAMRIXNAYMPKXQYD5ZJSHA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.174214444 (102h11m3.636306927s) [noise=true , address=89.149.201.147:4001 , piece_id=S4GZ34GDBSFVRS52KNUFSZFTPD4TDTFUMHUKWZSMNAHSFLZHZ5QQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.176187641 (102h11m3.638280114s) [noise=true , address=89.149.201.147:4023 , piece_id=HJKU46COZJYWALQCKMMZJ54AJG5SIIAN63X45HHVOTIUTL5GS2GA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.194676498 (102h11m3.656768971s) [noise=true , address=89.149.201.147:4024 , piece_id=RZP5QD6DULXOU2SP6XWPBTO7X6UTNNOVLAMYXACVOXA2JQH6ODEQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.552714451 (102h11m4.014806934s) [noise=true , address=81.171.14.70:4026 , piece_id=XRJKHK2QW5ZM7MAP2GEJMXALCFPXRNR7CQYOJHDMLJQNIVGRTI2A, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.578246218 (102h11m4.040338702s) [noise=true , address=81.171.14.70:4045 , piece_id=HESWYJWN53AC7JEYBA2LZJPE5N4E72MMVSAK4OOYRSUFKLPGS45Q, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.654575645 (102h11m4.116668128s) [noise=true , address=185.167.204.206:4040 , piece_id=IKSYLPGG2N2WNMYMJ2GE6LZOVS2ZLIWQDMO353TEOD52YDLQNFGQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.660837959 (102h11m4.122930442s) [noise=true , address=185.167.204.205:4013 , piece_id=AEOKIN7HCKU3S2QVUCXLL6M6ZG5ZFAE6MWAZGMIGQ46MFM5YTVFA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.684086459 (102h11m4.146178942s) [noise=true , address=185.167.204.206:4046 , piece_id=NW274SFLMGEFR6C4JRGTJ2V64PECG56KFU2233QXTSATRMQUFMYQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.717321825 (102h11m4.179414308s) [noise=true , address=185.167.204.205:4030 , piece_id=ED7OHHYUSNPOHSU3TF425LKVZKPNW55HAGMVD4YEOYZSTFUOC5QA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.734861794 (102h11m4.196954278s) [noise=true , address=185.167.204.205:4003 , piece_id=423DPR4A4F5ZIQCJ5PN73LVER7I5LJ33HC2PO2AU45M3F5W4KIZA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.743283301 (102h11m4.205375794s) [noise=true , address=185.167.204.205:4019 , piece_id=7IDC4PR2J5BSCO7LDYL4B5ZH72LERJ53QQ3CJQVEUSZBABQJFMRQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.765742828 (102h11m4.227835301s) [noise=true , address=185.167.204.205:4018 , piece_id=ODY5PWZUU2UQZCHUVZOT2AN6MWVDWWYLJO4UYTK2MMRVR3QVVLVA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.768463980 (102h11m4.230556463s) [noise=true , address=185.167.204.206:4051 , piece_id=62564CXV5ZNICNGEZMA4ZV4CZYHJW4NJZU64ARYG772SQ3MD5A2Q, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.770897814 (102h11m4.232990297s) [noise=true , address=185.167.204.205:4017 , piece_id=VOEPYLSGFS3F4TDYTKWHOR4KLIFUWIFKGTESNECVK5BTKAJQKPWQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.779253035 (102h11m4.241345518s) [noise=true , address=185.167.204.206:4068 , piece_id=WIE6MTMDCZI3OVGOPNC3D6GVLEP32IUUJQZLZJ6FPOEPM3HGESWQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.786276548 (102h11m4.248369031s) [noise=true , address=185.167.204.205:4028 , piece_id=W26MBIC7BOZU4WHU652JUMLGD6OMGR2FCZP77WUOU262AG3ULMUQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.786311094 (102h11m4.248403567s) [seg_pos={0 0}, upload=89929]: Segment reached optimal threshold of 26 pieces. Sep 27 19:42:18.786446752 (102h11m4.248539225s) [noise=true , address=185.167.204.205:4016 , piece_id=GYH5PXP335XI5C6BEZZ6IIFXI7CGHTRMX5JSGEOB25NRTEEUP46A, seg_pos={0 0}, upload=89929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ): protocol: expected piece hash; context canceled; EOF Sep 27 19:42:18.786454498 (102h11m4.248546981s) [noise=true , address=185.167.204.205:4016 , piece_id=GYH5PXP335XI5C6BEZZ6IIFXI7CGHTRMX5JSGEOB25NRTEEUP46A, seg_pos={0 0}, upload=89929]: Long tail cancellation due to optimal threshold Sep 27 19:42:18.786597260 (102h11m4.248689753s) [noise=true , address=185.167.204.206:4069 , piece_id=KGO3J74KNEVFROVWK4BP7FUE6C7HNGRPWNFUCRQQBQE5ZGQ4F27Q, seg_pos={0 0}, upload=89929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U): protocol: expected piece hash; context canceled; EOF Sep 27 19:42:18.786620463 (102h11m4.248712946s) [noise=true , address=185.167.204.206:4069 , piece_id=KGO3J74KNEVFROVWK4BP7FUE6C7HNGRPWNFUCRQQBQE5ZGQ4F27Q, seg_pos={0 0}, upload=89929]: Long tail cancellation due to optimal threshold Sep 27 19:42:18.786667804 (102h11m4.248760287s) [seg_pos={0 0}, upload=89929]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Sep 27 19:42:18.786777773 (102h11m4.248870246s) [upload=89929]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 19:42:18.786785097 (102h11m4.24887758s) [upload=89929]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 19:42:46.236891187 (102h11m31.69898367s) [upload=89929]: Done uploading segments... Sep 27 19:42:46.236928157 (102h11m31.69902064s) [upload=89929]: Upload finished. err: metaclient: manager closed: closed: read tcp 188.166.225.170:34184->35.207.121.91:7777: read: connection reset by peer Duration: 4.882278942s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 4.835608564s FAIL (2025-09-27 19:43:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: manager closed: closed: read tcp 188.166.225.170:34184->35.207.121.91:7777: read: connection reset by peer ======LOGS====== ploading piece... Sep 27 19:42:16.851904981 (102h11m2.313997454s) [noise=true , address=185.167.204.205:4019 , piece_id=7IDC4PR2J5BSCO7LDYL4B5ZH72LERJ53QQ3CJQVEUSZBABQJFMRQ, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.852399574 (102h11m2.314492047s) [noise=true , address=185.167.204.206:4069 , piece_id=KGO3J74KNEVFROVWK4BP7FUE6C7HNGRPWNFUCRQQBQE5ZGQ4F27Q, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.852873816 (102h11m2.314966299s) [noise=true , address=185.167.204.205:4016 , piece_id=GYH5PXP335XI5C6BEZZ6IIFXI7CGHTRMX5JSGEOB25NRTEEUP46A, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.853507803 (102h11m2.315600276s) [noise=true , address=185.167.204.206:4040 , piece_id=IKSYLPGG2N2WNMYMJ2GE6LZOVS2ZLIWQDMO353TEOD52YDLQNFGQ, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.855290378 (102h11m2.317382851s) [noise=true , address=81.171.14.70:4039 , piece_id=DI7DICLIQLQBNTR62K6KT2Z3F4PJMPEGRDAJJFHQNUF4D53WHQNQ, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.857320493 (102h11m2.319412967s) [noise=true , address=81.171.14.70:4028 , piece_id=TKDL5V3DXRB6IEX4UCDIORWH5C2JEA2SJV4FURSB2E43TUR5BVOQ, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.859477411 (102h11m2.321569884s) [noise=true , address=81.171.14.70:4036 , piece_id=WAGPANGKB4NLD75UKGEQM2V53LSKLZCAMRIXNAYMPKXQYD5ZJSHA, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.861648736 (102h11m2.323741219s) [noise=true , address=89.149.201.147:4001 , piece_id=S4GZ34GDBSFVRS52KNUFSZFTPD4TDTFUMHUKWZSMNAHSFLZHZ5QQ, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.863879203 (102h11m2.325971686s) [noise=true , address=185.167.204.206:4051 , piece_id=62564CXV5ZNICNGEZMA4ZV4CZYHJW4NJZU64ARYG772SQ3MD5A2Q, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.866136491 (102h11m2.328228974s) [noise=true , address=81.171.14.70:4045 , piece_id=HESWYJWN53AC7JEYBA2LZJPE5N4E72MMVSAK4OOYRSUFKLPGS45Q, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:17.717331390 (102h11m3.179423873s) [upload=89929]: Next returned nil segment Sep 27 19:42:17.717367178 (102h11m3.179459662s) [upload=89929]: Waiting for error group managing segments... Sep 27 19:42:18.087365918 (102h11m3.549458401s) [noise=true , address=81.171.14.70:4035 , piece_id=M3OI7SKEGYRTQCLXDWE364IABTZEDGP6QXWSQHHYGXTPP3VCLDZA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.105655594 (102h11m3.567748077s) [noise=true , address=81.171.14.70:4041 , piece_id=HESUR7NWPV427S2XRODHYHLS5KLNQLPOTPF3HOVNDQNRTUWGZ7KA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.107753430 (102h11m3.569845913s) [noise=true , address=81.171.14.70:4037 , piece_id=NF4KCWJHMWPWVF2SZLIHLFIJDH6TNCTDSQT3EQDBYMU6HHYBJ5OA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.114636617 (102h11m3.5767291s) [noise=true , address=81.171.14.70:4031 , piece_id=332OHLFNBAJ7LB36Y72MNALDU7NPXQIRY5QRVLAPJOTE6TCDOCEA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.125611034 (102h11m3.587703527s) [noise=true , address=81.171.14.70:4028 , piece_id=TKDL5V3DXRB6IEX4UCDIORWH5C2JEA2SJV4FURSB2E43TUR5BVOQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.130988554 (102h11m3.593081037s) [noise=true , address=81.171.14.70:4039 , piece_id=DI7DICLIQLQBNTR62K6KT2Z3F4PJMPEGRDAJJFHQNUF4D53WHQNQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.133905207 (102h11m3.59599769s) [noise=true , address=81.171.14.70:4047 , piece_id=2SEMSVDNNEQGCTKTJ2NPIXUAGFIHQ7B32O2FEOSFLYC55ZFWAE5Q, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.137235840 (102h11m3.599328313s) [noise=true , address=89.149.201.147:4019 , piece_id=DQJBIPFQLRH7PVC5ET7YSKJNQ4ZXT2BR6LHX74ZHRVHLIHEQXXUQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.138020274 (102h11m3.600112747s) [noise=true , address=89.149.201.147:4009 , piece_id=ZFTEYLRPQPK3MPAQS5OPXGZHJMAFOACLFRROWVIPXEZQFUP2ZGVQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.169014162 (102h11m3.631106645s) [noise=true , address=81.171.14.70:4036 , piece_id=WAGPANGKB4NLD75UKGEQM2V53LSKLZCAMRIXNAYMPKXQYD5ZJSHA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.174214444 (102h11m3.636306927s) [noise=true , address=89.149.201.147:4001 , piece_id=S4GZ34GDBSFVRS52KNUFSZFTPD4TDTFUMHUKWZSMNAHSFLZHZ5QQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.176187641 (102h11m3.638280114s) [noise=true , address=89.149.201.147:4023 , piece_id=HJKU46COZJYWALQCKMMZJ54AJG5SIIAN63X45HHVOTIUTL5GS2GA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.194676498 (102h11m3.656768971s) [noise=true , address=89.149.201.147:4024 , piece_id=RZP5QD6DULXOU2SP6XWPBTO7X6UTNNOVLAMYXACVOXA2JQH6ODEQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.552714451 (102h11m4.014806934s) [noise=true , address=81.171.14.70:4026 , piece_id=XRJKHK2QW5ZM7MAP2GEJMXALCFPXRNR7CQYOJHDMLJQNIVGRTI2A, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.578246218 (102h11m4.040338702s) [noise=true , address=81.171.14.70:4045 , piece_id=HESWYJWN53AC7JEYBA2LZJPE5N4E72MMVSAK4OOYRSUFKLPGS45Q, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.654575645 (102h11m4.116668128s) [noise=true , address=185.167.204.206:4040 , piece_id=IKSYLPGG2N2WNMYMJ2GE6LZOVS2ZLIWQDMO353TEOD52YDLQNFGQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.660837959 (102h11m4.122930442s) [noise=true , address=185.167.204.205:4013 , piece_id=AEOKIN7HCKU3S2QVUCXLL6M6ZG5ZFAE6MWAZGMIGQ46MFM5YTVFA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.684086459 (102h11m4.146178942s) [noise=true , address=185.167.204.206:4046 , piece_id=NW274SFLMGEFR6C4JRGTJ2V64PECG56KFU2233QXTSATRMQUFMYQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.717321825 (102h11m4.179414308s) [noise=true , address=185.167.204.205:4030 , piece_id=ED7OHHYUSNPOHSU3TF425LKVZKPNW55HAGMVD4YEOYZSTFUOC5QA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.734861794 (102h11m4.196954278s) [noise=true , address=185.167.204.205:4003 , piece_id=423DPR4A4F5ZIQCJ5PN73LVER7I5LJ33HC2PO2AU45M3F5W4KIZA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.743283301 (102h11m4.205375794s) [noise=true , address=185.167.204.205:4019 , piece_id=7IDC4PR2J5BSCO7LDYL4B5ZH72LERJ53QQ3CJQVEUSZBABQJFMRQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.765742828 (102h11m4.227835301s) [noise=true , address=185.167.204.205:4018 , piece_id=ODY5PWZUU2UQZCHUVZOT2AN6MWVDWWYLJO4UYTK2MMRVR3QVVLVA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.768463980 (102h11m4.230556463s) [noise=true , address=185.167.204.206:4051 , piece_id=62564CXV5ZNICNGEZMA4ZV4CZYHJW4NJZU64ARYG772SQ3MD5A2Q, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.770897814 (102h11m4.232990297s) [noise=true , address=185.167.204.205:4017 , piece_id=VOEPYLSGFS3F4TDYTKWHOR4KLIFUWIFKGTESNECVK5BTKAJQKPWQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.779253035 (102h11m4.241345518s) [noise=true , address=185.167.204.206:4068 , piece_id=WIE6MTMDCZI3OVGOPNC3D6GVLEP32IUUJQZLZJ6FPOEPM3HGESWQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.786276548 (102h11m4.248369031s) [noise=true , address=185.167.204.205:4028 , piece_id=W26MBIC7BOZU4WHU652JUMLGD6OMGR2FCZP77WUOU262AG3ULMUQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.786311094 (102h11m4.248403567s) [seg_pos={0 0}, upload=89929]: Segment reached optimal threshold of 26 pieces. Sep 27 19:42:18.786446752 (102h11m4.248539225s) [noise=true , address=185.167.204.205:4016 , piece_id=GYH5PXP335XI5C6BEZZ6IIFXI7CGHTRMX5JSGEOB25NRTEEUP46A, seg_pos={0 0}, upload=89929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ): protocol: expected piece hash; context canceled; EOF Sep 27 19:42:18.786454498 (102h11m4.248546981s) [noise=true , address=185.167.204.205:4016 , piece_id=GYH5PXP335XI5C6BEZZ6IIFXI7CGHTRMX5JSGEOB25NRTEEUP46A, seg_pos={0 0}, upload=89929]: Long tail cancellation due to optimal threshold Sep 27 19:42:18.786597260 (102h11m4.248689753s) [noise=true , address=185.167.204.206:4069 , piece_id=KGO3J74KNEVFROVWK4BP7FUE6C7HNGRPWNFUCRQQBQE5ZGQ4F27Q, seg_pos={0 0}, upload=89929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U): protocol: expected piece hash; context canceled; EOF Sep 27 19:42:18.786620463 (102h11m4.248712946s) [noise=true , address=185.167.204.206:4069 , piece_id=KGO3J74KNEVFROVWK4BP7FUE6C7HNGRPWNFUCRQQBQE5ZGQ4F27Q, seg_pos={0 0}, upload=89929]: Long tail cancellation due to optimal threshold Sep 27 19:42:18.786667804 (102h11m4.248760287s) [seg_pos={0 0}, upload=89929]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Sep 27 19:42:18.786777773 (102h11m4.248870246s) [upload=89929]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 19:42:18.786785097 (102h11m4.24887758s) [upload=89929]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 19:42:46.236891187 (102h11m31.69898367s) [upload=89929]: Done uploading segments... Sep 27 19:42:46.236928157 (102h11m31.69902064s) [upload=89929]: Upload finished. err: metaclient: manager closed: closed: read tcp 188.166.225.170:34184->35.207.121.91:7777: read: connection reset by peer Duration: 4.83564412s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............................X....X..X.X....................] duration: 6.545187755s FAIL (2025-09-27 19:43:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 27 19:27:22.886943307 (101h56m8.34903578s) [upload=89705]: Starting upload Sep 27 19:27:22.887049439 (101h56m8.349141922s) [upload=89705]: Done starting upload Sep 27 19:27:22.887160700 (101h56m8.349253184s) [upload=89705]: Uploading segments... Sep 27 19:27:22.887344050 (101h56m8.349436533s) [upload=89705]: Got next segment. Inline: false Sep 27 19:27:22.887393123 (101h56m8.349485606s) [upload=89705]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:27:22.887400948 (101h56m8.349493431s) [upload=89705]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:27:23.144152141 (101h56m8.606244624s) [seg_pos={0 0}, upload=89705]: Begin upload segment... Sep 27 19:27:23.144224288 (101h56m8.606316771s) [seg_pos={0 0}, upload=89705]: Done begin upload segment. Sep 27 19:27:23.144303519 (101h56m8.606396002s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144356600 (101h56m8.606449083s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144899694 (101h56m8.606992177s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145386381 (101h56m8.607478864s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145749413 (101h56m8.607841896s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146190593 (101h56m8.608283076s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146569114 (101h56m8.608661597s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146975998 (101h56m8.609068481s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.147437558 (101h56m8.609530051s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.148991526 (101h56m8.611083999s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.151370226 (101h56m8.613462699s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.153185462 (101h56m8.615277966s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.154735545 (101h56m8.616828028s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.158605123 (101h56m8.620697606s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162754133 (101h56m8.624846616s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162908967 (101h56m8.62500145s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:24.838034007 (101h56m10.30012649s) [upload=89705]: Next returned nil segment Sep 27 19:27:24.838098610 (101h56m10.300191093s) [upload=89705]: Waiting for error group managing segments... Sep 27 19:27:25.171482756 (101h56m10.633575229s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.176947972 (101h56m10.639040456s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.180229350 (101h56m10.642321834s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.181707686 (101h56m10.643800159s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.182762434 (101h56m10.644854917s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.192828732 (101h56m10.654921205s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.198888852 (101h56m10.660981335s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.205614519 (101h56m10.667707012s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.207989521 (101h56m10.670081994s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.222032181 (101h56m10.684124654s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.290677625 (101h56m10.752770118s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.652827506 (101h56m11.114919989s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:52.180260159 (101h56m37.642352652s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180423099 (101h56m37.642515582s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180455370 (101h56m37.642547853s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180565359 (101h56m37.642657842s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:37784->77.90.31.15:4697: use of closed network connection; conn closed: op Sep 27 19:27:52.180610105 (101h56m37.642702588s) [seg_pos={0 0}, upload=89705]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 27 19:27:52.433487771 (101h56m37.895580254s) [upload=89705]: Done uploading segments... Sep 27 19:27:52.433517006 (101h56m37.89560948s) [upload=89705]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.545223613s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 4.83211883s FAIL (2025-09-27 19:43:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: manager closed: closed: read tcp 188.166.225.170:34184->35.207.121.91:7777: read: connection reset by peer ======LOGS====== ploading piece... Sep 27 19:42:16.851904981 (102h11m2.313997454s) [noise=true , address=185.167.204.205:4019 , piece_id=7IDC4PR2J5BSCO7LDYL4B5ZH72LERJ53QQ3CJQVEUSZBABQJFMRQ, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.852399574 (102h11m2.314492047s) [noise=true , address=185.167.204.206:4069 , piece_id=KGO3J74KNEVFROVWK4BP7FUE6C7HNGRPWNFUCRQQBQE5ZGQ4F27Q, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.852873816 (102h11m2.314966299s) [noise=true , address=185.167.204.205:4016 , piece_id=GYH5PXP335XI5C6BEZZ6IIFXI7CGHTRMX5JSGEOB25NRTEEUP46A, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.853507803 (102h11m2.315600276s) [noise=true , address=185.167.204.206:4040 , piece_id=IKSYLPGG2N2WNMYMJ2GE6LZOVS2ZLIWQDMO353TEOD52YDLQNFGQ, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.855290378 (102h11m2.317382851s) [noise=true , address=81.171.14.70:4039 , piece_id=DI7DICLIQLQBNTR62K6KT2Z3F4PJMPEGRDAJJFHQNUF4D53WHQNQ, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.857320493 (102h11m2.319412967s) [noise=true , address=81.171.14.70:4028 , piece_id=TKDL5V3DXRB6IEX4UCDIORWH5C2JEA2SJV4FURSB2E43TUR5BVOQ, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.859477411 (102h11m2.321569884s) [noise=true , address=81.171.14.70:4036 , piece_id=WAGPANGKB4NLD75UKGEQM2V53LSKLZCAMRIXNAYMPKXQYD5ZJSHA, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.861648736 (102h11m2.323741219s) [noise=true , address=89.149.201.147:4001 , piece_id=S4GZ34GDBSFVRS52KNUFSZFTPD4TDTFUMHUKWZSMNAHSFLZHZ5QQ, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.863879203 (102h11m2.325971686s) [noise=true , address=185.167.204.206:4051 , piece_id=62564CXV5ZNICNGEZMA4ZV4CZYHJW4NJZU64ARYG772SQ3MD5A2Q, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.866136491 (102h11m2.328228974s) [noise=true , address=81.171.14.70:4045 , piece_id=HESWYJWN53AC7JEYBA2LZJPE5N4E72MMVSAK4OOYRSUFKLPGS45Q, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:17.717331390 (102h11m3.179423873s) [upload=89929]: Next returned nil segment Sep 27 19:42:17.717367178 (102h11m3.179459662s) [upload=89929]: Waiting for error group managing segments... Sep 27 19:42:18.087365918 (102h11m3.549458401s) [noise=true , address=81.171.14.70:4035 , piece_id=M3OI7SKEGYRTQCLXDWE364IABTZEDGP6QXWSQHHYGXTPP3VCLDZA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.105655594 (102h11m3.567748077s) [noise=true , address=81.171.14.70:4041 , piece_id=HESUR7NWPV427S2XRODHYHLS5KLNQLPOTPF3HOVNDQNRTUWGZ7KA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.107753430 (102h11m3.569845913s) [noise=true , address=81.171.14.70:4037 , piece_id=NF4KCWJHMWPWVF2SZLIHLFIJDH6TNCTDSQT3EQDBYMU6HHYBJ5OA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.114636617 (102h11m3.5767291s) [noise=true , address=81.171.14.70:4031 , piece_id=332OHLFNBAJ7LB36Y72MNALDU7NPXQIRY5QRVLAPJOTE6TCDOCEA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.125611034 (102h11m3.587703527s) [noise=true , address=81.171.14.70:4028 , piece_id=TKDL5V3DXRB6IEX4UCDIORWH5C2JEA2SJV4FURSB2E43TUR5BVOQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.130988554 (102h11m3.593081037s) [noise=true , address=81.171.14.70:4039 , piece_id=DI7DICLIQLQBNTR62K6KT2Z3F4PJMPEGRDAJJFHQNUF4D53WHQNQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.133905207 (102h11m3.59599769s) [noise=true , address=81.171.14.70:4047 , piece_id=2SEMSVDNNEQGCTKTJ2NPIXUAGFIHQ7B32O2FEOSFLYC55ZFWAE5Q, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.137235840 (102h11m3.599328313s) [noise=true , address=89.149.201.147:4019 , piece_id=DQJBIPFQLRH7PVC5ET7YSKJNQ4ZXT2BR6LHX74ZHRVHLIHEQXXUQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.138020274 (102h11m3.600112747s) [noise=true , address=89.149.201.147:4009 , piece_id=ZFTEYLRPQPK3MPAQS5OPXGZHJMAFOACLFRROWVIPXEZQFUP2ZGVQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.169014162 (102h11m3.631106645s) [noise=true , address=81.171.14.70:4036 , piece_id=WAGPANGKB4NLD75UKGEQM2V53LSKLZCAMRIXNAYMPKXQYD5ZJSHA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.174214444 (102h11m3.636306927s) [noise=true , address=89.149.201.147:4001 , piece_id=S4GZ34GDBSFVRS52KNUFSZFTPD4TDTFUMHUKWZSMNAHSFLZHZ5QQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.176187641 (102h11m3.638280114s) [noise=true , address=89.149.201.147:4023 , piece_id=HJKU46COZJYWALQCKMMZJ54AJG5SIIAN63X45HHVOTIUTL5GS2GA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.194676498 (102h11m3.656768971s) [noise=true , address=89.149.201.147:4024 , piece_id=RZP5QD6DULXOU2SP6XWPBTO7X6UTNNOVLAMYXACVOXA2JQH6ODEQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.552714451 (102h11m4.014806934s) [noise=true , address=81.171.14.70:4026 , piece_id=XRJKHK2QW5ZM7MAP2GEJMXALCFPXRNR7CQYOJHDMLJQNIVGRTI2A, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.578246218 (102h11m4.040338702s) [noise=true , address=81.171.14.70:4045 , piece_id=HESWYJWN53AC7JEYBA2LZJPE5N4E72MMVSAK4OOYRSUFKLPGS45Q, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.654575645 (102h11m4.116668128s) [noise=true , address=185.167.204.206:4040 , piece_id=IKSYLPGG2N2WNMYMJ2GE6LZOVS2ZLIWQDMO353TEOD52YDLQNFGQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.660837959 (102h11m4.122930442s) [noise=true , address=185.167.204.205:4013 , piece_id=AEOKIN7HCKU3S2QVUCXLL6M6ZG5ZFAE6MWAZGMIGQ46MFM5YTVFA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.684086459 (102h11m4.146178942s) [noise=true , address=185.167.204.206:4046 , piece_id=NW274SFLMGEFR6C4JRGTJ2V64PECG56KFU2233QXTSATRMQUFMYQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.717321825 (102h11m4.179414308s) [noise=true , address=185.167.204.205:4030 , piece_id=ED7OHHYUSNPOHSU3TF425LKVZKPNW55HAGMVD4YEOYZSTFUOC5QA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.734861794 (102h11m4.196954278s) [noise=true , address=185.167.204.205:4003 , piece_id=423DPR4A4F5ZIQCJ5PN73LVER7I5LJ33HC2PO2AU45M3F5W4KIZA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.743283301 (102h11m4.205375794s) [noise=true , address=185.167.204.205:4019 , piece_id=7IDC4PR2J5BSCO7LDYL4B5ZH72LERJ53QQ3CJQVEUSZBABQJFMRQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.765742828 (102h11m4.227835301s) [noise=true , address=185.167.204.205:4018 , piece_id=ODY5PWZUU2UQZCHUVZOT2AN6MWVDWWYLJO4UYTK2MMRVR3QVVLVA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.768463980 (102h11m4.230556463s) [noise=true , address=185.167.204.206:4051 , piece_id=62564CXV5ZNICNGEZMA4ZV4CZYHJW4NJZU64ARYG772SQ3MD5A2Q, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.770897814 (102h11m4.232990297s) [noise=true , address=185.167.204.205:4017 , piece_id=VOEPYLSGFS3F4TDYTKWHOR4KLIFUWIFKGTESNECVK5BTKAJQKPWQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.779253035 (102h11m4.241345518s) [noise=true , address=185.167.204.206:4068 , piece_id=WIE6MTMDCZI3OVGOPNC3D6GVLEP32IUUJQZLZJ6FPOEPM3HGESWQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.786276548 (102h11m4.248369031s) [noise=true , address=185.167.204.205:4028 , piece_id=W26MBIC7BOZU4WHU652JUMLGD6OMGR2FCZP77WUOU262AG3ULMUQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.786311094 (102h11m4.248403567s) [seg_pos={0 0}, upload=89929]: Segment reached optimal threshold of 26 pieces. Sep 27 19:42:18.786446752 (102h11m4.248539225s) [noise=true , address=185.167.204.205:4016 , piece_id=GYH5PXP335XI5C6BEZZ6IIFXI7CGHTRMX5JSGEOB25NRTEEUP46A, seg_pos={0 0}, upload=89929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ): protocol: expected piece hash; context canceled; EOF Sep 27 19:42:18.786454498 (102h11m4.248546981s) [noise=true , address=185.167.204.205:4016 , piece_id=GYH5PXP335XI5C6BEZZ6IIFXI7CGHTRMX5JSGEOB25NRTEEUP46A, seg_pos={0 0}, upload=89929]: Long tail cancellation due to optimal threshold Sep 27 19:42:18.786597260 (102h11m4.248689753s) [noise=true , address=185.167.204.206:4069 , piece_id=KGO3J74KNEVFROVWK4BP7FUE6C7HNGRPWNFUCRQQBQE5ZGQ4F27Q, seg_pos={0 0}, upload=89929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U): protocol: expected piece hash; context canceled; EOF Sep 27 19:42:18.786620463 (102h11m4.248712946s) [noise=true , address=185.167.204.206:4069 , piece_id=KGO3J74KNEVFROVWK4BP7FUE6C7HNGRPWNFUCRQQBQE5ZGQ4F27Q, seg_pos={0 0}, upload=89929]: Long tail cancellation due to optimal threshold Sep 27 19:42:18.786667804 (102h11m4.248760287s) [seg_pos={0 0}, upload=89929]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Sep 27 19:42:18.786777773 (102h11m4.248870246s) [upload=89929]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 19:42:18.786785097 (102h11m4.24887758s) [upload=89929]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 19:42:46.236891187 (102h11m31.69898367s) [upload=89929]: Done uploading segments... Sep 27 19:42:46.236928157 (102h11m31.69902064s) [upload=89929]: Upload finished. err: metaclient: manager closed: closed: read tcp 188.166.225.170:34184->35.207.121.91:7777: read: connection reset by peer Duration: 4.832155701s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............................X....X..X.X...................] duration: 5.943839068s FAIL (2025-09-27 19:42:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 27 19:27:22.886943307 (101h56m8.34903578s) [upload=89705]: Starting upload Sep 27 19:27:22.887049439 (101h56m8.349141922s) [upload=89705]: Done starting upload Sep 27 19:27:22.887160700 (101h56m8.349253184s) [upload=89705]: Uploading segments... Sep 27 19:27:22.887344050 (101h56m8.349436533s) [upload=89705]: Got next segment. Inline: false Sep 27 19:27:22.887393123 (101h56m8.349485606s) [upload=89705]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:27:22.887400948 (101h56m8.349493431s) [upload=89705]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:27:23.144152141 (101h56m8.606244624s) [seg_pos={0 0}, upload=89705]: Begin upload segment... Sep 27 19:27:23.144224288 (101h56m8.606316771s) [seg_pos={0 0}, upload=89705]: Done begin upload segment. Sep 27 19:27:23.144303519 (101h56m8.606396002s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144356600 (101h56m8.606449083s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144899694 (101h56m8.606992177s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145386381 (101h56m8.607478864s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145749413 (101h56m8.607841896s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146190593 (101h56m8.608283076s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146569114 (101h56m8.608661597s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146975998 (101h56m8.609068481s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.147437558 (101h56m8.609530051s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.148991526 (101h56m8.611083999s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.151370226 (101h56m8.613462699s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.153185462 (101h56m8.615277966s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.154735545 (101h56m8.616828028s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.158605123 (101h56m8.620697606s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162754133 (101h56m8.624846616s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162908967 (101h56m8.62500145s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:24.838034007 (101h56m10.30012649s) [upload=89705]: Next returned nil segment Sep 27 19:27:24.838098610 (101h56m10.300191093s) [upload=89705]: Waiting for error group managing segments... Sep 27 19:27:25.171482756 (101h56m10.633575229s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.176947972 (101h56m10.639040456s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.180229350 (101h56m10.642321834s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.181707686 (101h56m10.643800159s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.182762434 (101h56m10.644854917s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.192828732 (101h56m10.654921205s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.198888852 (101h56m10.660981335s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.205614519 (101h56m10.667707012s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.207989521 (101h56m10.670081994s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.222032181 (101h56m10.684124654s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.290677625 (101h56m10.752770118s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.652827506 (101h56m11.114919989s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:52.180260159 (101h56m37.642352652s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180423099 (101h56m37.642515582s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180455370 (101h56m37.642547853s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180565359 (101h56m37.642657842s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:37784->77.90.31.15:4697: use of closed network connection; conn closed: op Sep 27 19:27:52.180610105 (101h56m37.642702588s) [seg_pos={0 0}, upload=89705]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 27 19:27:52.433487771 (101h56m37.895580254s) [upload=89705]: Done uploading segments... Sep 27 19:27:52.433517006 (101h56m37.89560948s) [upload=89705]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.943874015s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 4.924682034s FAIL (2025-09-27 19:42:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: manager closed: closed: read tcp 188.166.225.170:34184->35.207.121.91:7777: read: connection reset by peer ======LOGS====== ploading piece... Sep 27 19:42:16.851904981 (102h11m2.313997454s) [noise=true , address=185.167.204.205:4019 , piece_id=7IDC4PR2J5BSCO7LDYL4B5ZH72LERJ53QQ3CJQVEUSZBABQJFMRQ, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.852399574 (102h11m2.314492047s) [noise=true , address=185.167.204.206:4069 , piece_id=KGO3J74KNEVFROVWK4BP7FUE6C7HNGRPWNFUCRQQBQE5ZGQ4F27Q, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.852873816 (102h11m2.314966299s) [noise=true , address=185.167.204.205:4016 , piece_id=GYH5PXP335XI5C6BEZZ6IIFXI7CGHTRMX5JSGEOB25NRTEEUP46A, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.853507803 (102h11m2.315600276s) [noise=true , address=185.167.204.206:4040 , piece_id=IKSYLPGG2N2WNMYMJ2GE6LZOVS2ZLIWQDMO353TEOD52YDLQNFGQ, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.855290378 (102h11m2.317382851s) [noise=true , address=81.171.14.70:4039 , piece_id=DI7DICLIQLQBNTR62K6KT2Z3F4PJMPEGRDAJJFHQNUF4D53WHQNQ, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.857320493 (102h11m2.319412967s) [noise=true , address=81.171.14.70:4028 , piece_id=TKDL5V3DXRB6IEX4UCDIORWH5C2JEA2SJV4FURSB2E43TUR5BVOQ, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.859477411 (102h11m2.321569884s) [noise=true , address=81.171.14.70:4036 , piece_id=WAGPANGKB4NLD75UKGEQM2V53LSKLZCAMRIXNAYMPKXQYD5ZJSHA, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.861648736 (102h11m2.323741219s) [noise=true , address=89.149.201.147:4001 , piece_id=S4GZ34GDBSFVRS52KNUFSZFTPD4TDTFUMHUKWZSMNAHSFLZHZ5QQ, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.863879203 (102h11m2.325971686s) [noise=true , address=185.167.204.206:4051 , piece_id=62564CXV5ZNICNGEZMA4ZV4CZYHJW4NJZU64ARYG772SQ3MD5A2Q, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.866136491 (102h11m2.328228974s) [noise=true , address=81.171.14.70:4045 , piece_id=HESWYJWN53AC7JEYBA2LZJPE5N4E72MMVSAK4OOYRSUFKLPGS45Q, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:17.717331390 (102h11m3.179423873s) [upload=89929]: Next returned nil segment Sep 27 19:42:17.717367178 (102h11m3.179459662s) [upload=89929]: Waiting for error group managing segments... Sep 27 19:42:18.087365918 (102h11m3.549458401s) [noise=true , address=81.171.14.70:4035 , piece_id=M3OI7SKEGYRTQCLXDWE364IABTZEDGP6QXWSQHHYGXTPP3VCLDZA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.105655594 (102h11m3.567748077s) [noise=true , address=81.171.14.70:4041 , piece_id=HESUR7NWPV427S2XRODHYHLS5KLNQLPOTPF3HOVNDQNRTUWGZ7KA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.107753430 (102h11m3.569845913s) [noise=true , address=81.171.14.70:4037 , piece_id=NF4KCWJHMWPWVF2SZLIHLFIJDH6TNCTDSQT3EQDBYMU6HHYBJ5OA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.114636617 (102h11m3.5767291s) [noise=true , address=81.171.14.70:4031 , piece_id=332OHLFNBAJ7LB36Y72MNALDU7NPXQIRY5QRVLAPJOTE6TCDOCEA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.125611034 (102h11m3.587703527s) [noise=true , address=81.171.14.70:4028 , piece_id=TKDL5V3DXRB6IEX4UCDIORWH5C2JEA2SJV4FURSB2E43TUR5BVOQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.130988554 (102h11m3.593081037s) [noise=true , address=81.171.14.70:4039 , piece_id=DI7DICLIQLQBNTR62K6KT2Z3F4PJMPEGRDAJJFHQNUF4D53WHQNQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.133905207 (102h11m3.59599769s) [noise=true , address=81.171.14.70:4047 , piece_id=2SEMSVDNNEQGCTKTJ2NPIXUAGFIHQ7B32O2FEOSFLYC55ZFWAE5Q, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.137235840 (102h11m3.599328313s) [noise=true , address=89.149.201.147:4019 , piece_id=DQJBIPFQLRH7PVC5ET7YSKJNQ4ZXT2BR6LHX74ZHRVHLIHEQXXUQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.138020274 (102h11m3.600112747s) [noise=true , address=89.149.201.147:4009 , piece_id=ZFTEYLRPQPK3MPAQS5OPXGZHJMAFOACLFRROWVIPXEZQFUP2ZGVQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.169014162 (102h11m3.631106645s) [noise=true , address=81.171.14.70:4036 , piece_id=WAGPANGKB4NLD75UKGEQM2V53LSKLZCAMRIXNAYMPKXQYD5ZJSHA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.174214444 (102h11m3.636306927s) [noise=true , address=89.149.201.147:4001 , piece_id=S4GZ34GDBSFVRS52KNUFSZFTPD4TDTFUMHUKWZSMNAHSFLZHZ5QQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.176187641 (102h11m3.638280114s) [noise=true , address=89.149.201.147:4023 , piece_id=HJKU46COZJYWALQCKMMZJ54AJG5SIIAN63X45HHVOTIUTL5GS2GA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.194676498 (102h11m3.656768971s) [noise=true , address=89.149.201.147:4024 , piece_id=RZP5QD6DULXOU2SP6XWPBTO7X6UTNNOVLAMYXACVOXA2JQH6ODEQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.552714451 (102h11m4.014806934s) [noise=true , address=81.171.14.70:4026 , piece_id=XRJKHK2QW5ZM7MAP2GEJMXALCFPXRNR7CQYOJHDMLJQNIVGRTI2A, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.578246218 (102h11m4.040338702s) [noise=true , address=81.171.14.70:4045 , piece_id=HESWYJWN53AC7JEYBA2LZJPE5N4E72MMVSAK4OOYRSUFKLPGS45Q, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.654575645 (102h11m4.116668128s) [noise=true , address=185.167.204.206:4040 , piece_id=IKSYLPGG2N2WNMYMJ2GE6LZOVS2ZLIWQDMO353TEOD52YDLQNFGQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.660837959 (102h11m4.122930442s) [noise=true , address=185.167.204.205:4013 , piece_id=AEOKIN7HCKU3S2QVUCXLL6M6ZG5ZFAE6MWAZGMIGQ46MFM5YTVFA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.684086459 (102h11m4.146178942s) [noise=true , address=185.167.204.206:4046 , piece_id=NW274SFLMGEFR6C4JRGTJ2V64PECG56KFU2233QXTSATRMQUFMYQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.717321825 (102h11m4.179414308s) [noise=true , address=185.167.204.205:4030 , piece_id=ED7OHHYUSNPOHSU3TF425LKVZKPNW55HAGMVD4YEOYZSTFUOC5QA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.734861794 (102h11m4.196954278s) [noise=true , address=185.167.204.205:4003 , piece_id=423DPR4A4F5ZIQCJ5PN73LVER7I5LJ33HC2PO2AU45M3F5W4KIZA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.743283301 (102h11m4.205375794s) [noise=true , address=185.167.204.205:4019 , piece_id=7IDC4PR2J5BSCO7LDYL4B5ZH72LERJ53QQ3CJQVEUSZBABQJFMRQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.765742828 (102h11m4.227835301s) [noise=true , address=185.167.204.205:4018 , piece_id=ODY5PWZUU2UQZCHUVZOT2AN6MWVDWWYLJO4UYTK2MMRVR3QVVLVA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.768463980 (102h11m4.230556463s) [noise=true , address=185.167.204.206:4051 , piece_id=62564CXV5ZNICNGEZMA4ZV4CZYHJW4NJZU64ARYG772SQ3MD5A2Q, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.770897814 (102h11m4.232990297s) [noise=true , address=185.167.204.205:4017 , piece_id=VOEPYLSGFS3F4TDYTKWHOR4KLIFUWIFKGTESNECVK5BTKAJQKPWQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.779253035 (102h11m4.241345518s) [noise=true , address=185.167.204.206:4068 , piece_id=WIE6MTMDCZI3OVGOPNC3D6GVLEP32IUUJQZLZJ6FPOEPM3HGESWQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.786276548 (102h11m4.248369031s) [noise=true , address=185.167.204.205:4028 , piece_id=W26MBIC7BOZU4WHU652JUMLGD6OMGR2FCZP77WUOU262AG3ULMUQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.786311094 (102h11m4.248403567s) [seg_pos={0 0}, upload=89929]: Segment reached optimal threshold of 26 pieces. Sep 27 19:42:18.786446752 (102h11m4.248539225s) [noise=true , address=185.167.204.205:4016 , piece_id=GYH5PXP335XI5C6BEZZ6IIFXI7CGHTRMX5JSGEOB25NRTEEUP46A, seg_pos={0 0}, upload=89929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ): protocol: expected piece hash; context canceled; EOF Sep 27 19:42:18.786454498 (102h11m4.248546981s) [noise=true , address=185.167.204.205:4016 , piece_id=GYH5PXP335XI5C6BEZZ6IIFXI7CGHTRMX5JSGEOB25NRTEEUP46A, seg_pos={0 0}, upload=89929]: Long tail cancellation due to optimal threshold Sep 27 19:42:18.786597260 (102h11m4.248689753s) [noise=true , address=185.167.204.206:4069 , piece_id=KGO3J74KNEVFROVWK4BP7FUE6C7HNGRPWNFUCRQQBQE5ZGQ4F27Q, seg_pos={0 0}, upload=89929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U): protocol: expected piece hash; context canceled; EOF Sep 27 19:42:18.786620463 (102h11m4.248712946s) [noise=true , address=185.167.204.206:4069 , piece_id=KGO3J74KNEVFROVWK4BP7FUE6C7HNGRPWNFUCRQQBQE5ZGQ4F27Q, seg_pos={0 0}, upload=89929]: Long tail cancellation due to optimal threshold Sep 27 19:42:18.786667804 (102h11m4.248760287s) [seg_pos={0 0}, upload=89929]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Sep 27 19:42:18.786777773 (102h11m4.248870246s) [upload=89929]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 19:42:18.786785097 (102h11m4.24887758s) [upload=89929]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 19:42:46.236891187 (102h11m31.69898367s) [upload=89929]: Done uploading segments... Sep 27 19:42:46.236928157 (102h11m31.69902064s) [upload=89929]: Upload finished. err: metaclient: manager closed: closed: read tcp 188.166.225.170:34184->35.207.121.91:7777: read: connection reset by peer Duration: 4.924701092s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.110837916s FAIL (2025-09-27 19:42:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: manager closed: closed: read tcp 188.166.225.170:34184->35.207.121.91:7777: read: connection reset by peer ======LOGS====== ploading piece... Sep 27 19:42:16.851904981 (102h11m2.313997454s) [noise=true , address=185.167.204.205:4019 , piece_id=7IDC4PR2J5BSCO7LDYL4B5ZH72LERJ53QQ3CJQVEUSZBABQJFMRQ, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.852399574 (102h11m2.314492047s) [noise=true , address=185.167.204.206:4069 , piece_id=KGO3J74KNEVFROVWK4BP7FUE6C7HNGRPWNFUCRQQBQE5ZGQ4F27Q, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.852873816 (102h11m2.314966299s) [noise=true , address=185.167.204.205:4016 , piece_id=GYH5PXP335XI5C6BEZZ6IIFXI7CGHTRMX5JSGEOB25NRTEEUP46A, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.853507803 (102h11m2.315600276s) [noise=true , address=185.167.204.206:4040 , piece_id=IKSYLPGG2N2WNMYMJ2GE6LZOVS2ZLIWQDMO353TEOD52YDLQNFGQ, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.855290378 (102h11m2.317382851s) [noise=true , address=81.171.14.70:4039 , piece_id=DI7DICLIQLQBNTR62K6KT2Z3F4PJMPEGRDAJJFHQNUF4D53WHQNQ, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.857320493 (102h11m2.319412967s) [noise=true , address=81.171.14.70:4028 , piece_id=TKDL5V3DXRB6IEX4UCDIORWH5C2JEA2SJV4FURSB2E43TUR5BVOQ, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.859477411 (102h11m2.321569884s) [noise=true , address=81.171.14.70:4036 , piece_id=WAGPANGKB4NLD75UKGEQM2V53LSKLZCAMRIXNAYMPKXQYD5ZJSHA, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.861648736 (102h11m2.323741219s) [noise=true , address=89.149.201.147:4001 , piece_id=S4GZ34GDBSFVRS52KNUFSZFTPD4TDTFUMHUKWZSMNAHSFLZHZ5QQ, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.863879203 (102h11m2.325971686s) [noise=true , address=185.167.204.206:4051 , piece_id=62564CXV5ZNICNGEZMA4ZV4CZYHJW4NJZU64ARYG772SQ3MD5A2Q, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:16.866136491 (102h11m2.328228974s) [noise=true , address=81.171.14.70:4045 , piece_id=HESWYJWN53AC7JEYBA2LZJPE5N4E72MMVSAK4OOYRSUFKLPGS45Q, seg_pos={0 0}, upload=89929]: Uploading piece... Sep 27 19:42:17.717331390 (102h11m3.179423873s) [upload=89929]: Next returned nil segment Sep 27 19:42:17.717367178 (102h11m3.179459662s) [upload=89929]: Waiting for error group managing segments... Sep 27 19:42:18.087365918 (102h11m3.549458401s) [noise=true , address=81.171.14.70:4035 , piece_id=M3OI7SKEGYRTQCLXDWE364IABTZEDGP6QXWSQHHYGXTPP3VCLDZA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.105655594 (102h11m3.567748077s) [noise=true , address=81.171.14.70:4041 , piece_id=HESUR7NWPV427S2XRODHYHLS5KLNQLPOTPF3HOVNDQNRTUWGZ7KA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.107753430 (102h11m3.569845913s) [noise=true , address=81.171.14.70:4037 , piece_id=NF4KCWJHMWPWVF2SZLIHLFIJDH6TNCTDSQT3EQDBYMU6HHYBJ5OA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.114636617 (102h11m3.5767291s) [noise=true , address=81.171.14.70:4031 , piece_id=332OHLFNBAJ7LB36Y72MNALDU7NPXQIRY5QRVLAPJOTE6TCDOCEA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.125611034 (102h11m3.587703527s) [noise=true , address=81.171.14.70:4028 , piece_id=TKDL5V3DXRB6IEX4UCDIORWH5C2JEA2SJV4FURSB2E43TUR5BVOQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.130988554 (102h11m3.593081037s) [noise=true , address=81.171.14.70:4039 , piece_id=DI7DICLIQLQBNTR62K6KT2Z3F4PJMPEGRDAJJFHQNUF4D53WHQNQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.133905207 (102h11m3.59599769s) [noise=true , address=81.171.14.70:4047 , piece_id=2SEMSVDNNEQGCTKTJ2NPIXUAGFIHQ7B32O2FEOSFLYC55ZFWAE5Q, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.137235840 (102h11m3.599328313s) [noise=true , address=89.149.201.147:4019 , piece_id=DQJBIPFQLRH7PVC5ET7YSKJNQ4ZXT2BR6LHX74ZHRVHLIHEQXXUQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.138020274 (102h11m3.600112747s) [noise=true , address=89.149.201.147:4009 , piece_id=ZFTEYLRPQPK3MPAQS5OPXGZHJMAFOACLFRROWVIPXEZQFUP2ZGVQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.169014162 (102h11m3.631106645s) [noise=true , address=81.171.14.70:4036 , piece_id=WAGPANGKB4NLD75UKGEQM2V53LSKLZCAMRIXNAYMPKXQYD5ZJSHA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.174214444 (102h11m3.636306927s) [noise=true , address=89.149.201.147:4001 , piece_id=S4GZ34GDBSFVRS52KNUFSZFTPD4TDTFUMHUKWZSMNAHSFLZHZ5QQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.176187641 (102h11m3.638280114s) [noise=true , address=89.149.201.147:4023 , piece_id=HJKU46COZJYWALQCKMMZJ54AJG5SIIAN63X45HHVOTIUTL5GS2GA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.194676498 (102h11m3.656768971s) [noise=true , address=89.149.201.147:4024 , piece_id=RZP5QD6DULXOU2SP6XWPBTO7X6UTNNOVLAMYXACVOXA2JQH6ODEQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.552714451 (102h11m4.014806934s) [noise=true , address=81.171.14.70:4026 , piece_id=XRJKHK2QW5ZM7MAP2GEJMXALCFPXRNR7CQYOJHDMLJQNIVGRTI2A, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.578246218 (102h11m4.040338702s) [noise=true , address=81.171.14.70:4045 , piece_id=HESWYJWN53AC7JEYBA2LZJPE5N4E72MMVSAK4OOYRSUFKLPGS45Q, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.654575645 (102h11m4.116668128s) [noise=true , address=185.167.204.206:4040 , piece_id=IKSYLPGG2N2WNMYMJ2GE6LZOVS2ZLIWQDMO353TEOD52YDLQNFGQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.660837959 (102h11m4.122930442s) [noise=true , address=185.167.204.205:4013 , piece_id=AEOKIN7HCKU3S2QVUCXLL6M6ZG5ZFAE6MWAZGMIGQ46MFM5YTVFA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.684086459 (102h11m4.146178942s) [noise=true , address=185.167.204.206:4046 , piece_id=NW274SFLMGEFR6C4JRGTJ2V64PECG56KFU2233QXTSATRMQUFMYQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.717321825 (102h11m4.179414308s) [noise=true , address=185.167.204.205:4030 , piece_id=ED7OHHYUSNPOHSU3TF425LKVZKPNW55HAGMVD4YEOYZSTFUOC5QA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.734861794 (102h11m4.196954278s) [noise=true , address=185.167.204.205:4003 , piece_id=423DPR4A4F5ZIQCJ5PN73LVER7I5LJ33HC2PO2AU45M3F5W4KIZA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.743283301 (102h11m4.205375794s) [noise=true , address=185.167.204.205:4019 , piece_id=7IDC4PR2J5BSCO7LDYL4B5ZH72LERJ53QQ3CJQVEUSZBABQJFMRQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.765742828 (102h11m4.227835301s) [noise=true , address=185.167.204.205:4018 , piece_id=ODY5PWZUU2UQZCHUVZOT2AN6MWVDWWYLJO4UYTK2MMRVR3QVVLVA, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.768463980 (102h11m4.230556463s) [noise=true , address=185.167.204.206:4051 , piece_id=62564CXV5ZNICNGEZMA4ZV4CZYHJW4NJZU64ARYG772SQ3MD5A2Q, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.770897814 (102h11m4.232990297s) [noise=true , address=185.167.204.205:4017 , piece_id=VOEPYLSGFS3F4TDYTKWHOR4KLIFUWIFKGTESNECVK5BTKAJQKPWQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.779253035 (102h11m4.241345518s) [noise=true , address=185.167.204.206:4068 , piece_id=WIE6MTMDCZI3OVGOPNC3D6GVLEP32IUUJQZLZJ6FPOEPM3HGESWQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.786276548 (102h11m4.248369031s) [noise=true , address=185.167.204.205:4028 , piece_id=W26MBIC7BOZU4WHU652JUMLGD6OMGR2FCZP77WUOU262AG3ULMUQ, seg_pos={0 0}, upload=89929]: Done uploading piece. err: <nil> Sep 27 19:42:18.786311094 (102h11m4.248403567s) [seg_pos={0 0}, upload=89929]: Segment reached optimal threshold of 26 pieces. Sep 27 19:42:18.786446752 (102h11m4.248539225s) [noise=true , address=185.167.204.205:4016 , piece_id=GYH5PXP335XI5C6BEZZ6IIFXI7CGHTRMX5JSGEOB25NRTEEUP46A, seg_pos={0 0}, upload=89929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1scSMv7XDKsGKUMR8t1AdrQubP2142yorfVKaE4UyQoufi34TJ): protocol: expected piece hash; context canceled; EOF Sep 27 19:42:18.786454498 (102h11m4.248546981s) [noise=true , address=185.167.204.205:4016 , piece_id=GYH5PXP335XI5C6BEZZ6IIFXI7CGHTRMX5JSGEOB25NRTEEUP46A, seg_pos={0 0}, upload=89929]: Long tail cancellation due to optimal threshold Sep 27 19:42:18.786597260 (102h11m4.248689753s) [noise=true , address=185.167.204.206:4069 , piece_id=KGO3J74KNEVFROVWK4BP7FUE6C7HNGRPWNFUCRQQBQE5ZGQ4F27Q, seg_pos={0 0}, upload=89929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U): protocol: expected piece hash; context canceled; EOF Sep 27 19:42:18.786620463 (102h11m4.248712946s) [noise=true , address=185.167.204.206:4069 , piece_id=KGO3J74KNEVFROVWK4BP7FUE6C7HNGRPWNFUCRQQBQE5ZGQ4F27Q, seg_pos={0 0}, upload=89929]: Long tail cancellation due to optimal threshold Sep 27 19:42:18.786667804 (102h11m4.248760287s) [seg_pos={0 0}, upload=89929]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Sep 27 19:42:18.786777773 (102h11m4.248870246s) [upload=89929]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 19:42:18.786785097 (102h11m4.24887758s) [upload=89929]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 19:42:46.236891187 (102h11m31.69898367s) [upload=89929]: Done uploading segments... Sep 27 19:42:46.236928157 (102h11m31.69902064s) [upload=89929]: Upload finished. err: metaclient: manager closed: closed: read tcp 188.166.225.170:34184->35.207.121.91:7777: read: connection reset by peer Duration: 30.110865601s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............................X....X..X.X..................] duration: 6.127491748s FAIL (2025-09-27 19:41:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 27 19:27:22.886943307 (101h56m8.34903578s) [upload=89705]: Starting upload Sep 27 19:27:22.887049439 (101h56m8.349141922s) [upload=89705]: Done starting upload Sep 27 19:27:22.887160700 (101h56m8.349253184s) [upload=89705]: Uploading segments... Sep 27 19:27:22.887344050 (101h56m8.349436533s) [upload=89705]: Got next segment. Inline: false Sep 27 19:27:22.887393123 (101h56m8.349485606s) [upload=89705]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:27:22.887400948 (101h56m8.349493431s) [upload=89705]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:27:23.144152141 (101h56m8.606244624s) [seg_pos={0 0}, upload=89705]: Begin upload segment... Sep 27 19:27:23.144224288 (101h56m8.606316771s) [seg_pos={0 0}, upload=89705]: Done begin upload segment. Sep 27 19:27:23.144303519 (101h56m8.606396002s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144356600 (101h56m8.606449083s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144899694 (101h56m8.606992177s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145386381 (101h56m8.607478864s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145749413 (101h56m8.607841896s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146190593 (101h56m8.608283076s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146569114 (101h56m8.608661597s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146975998 (101h56m8.609068481s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.147437558 (101h56m8.609530051s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.148991526 (101h56m8.611083999s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.151370226 (101h56m8.613462699s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.153185462 (101h56m8.615277966s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.154735545 (101h56m8.616828028s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.158605123 (101h56m8.620697606s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162754133 (101h56m8.624846616s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162908967 (101h56m8.62500145s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:24.838034007 (101h56m10.30012649s) [upload=89705]: Next returned nil segment Sep 27 19:27:24.838098610 (101h56m10.300191093s) [upload=89705]: Waiting for error group managing segments... Sep 27 19:27:25.171482756 (101h56m10.633575229s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.176947972 (101h56m10.639040456s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.180229350 (101h56m10.642321834s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.181707686 (101h56m10.643800159s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.182762434 (101h56m10.644854917s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.192828732 (101h56m10.654921205s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.198888852 (101h56m10.660981335s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.205614519 (101h56m10.667707012s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.207989521 (101h56m10.670081994s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.222032181 (101h56m10.684124654s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.290677625 (101h56m10.752770118s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.652827506 (101h56m11.114919989s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:52.180260159 (101h56m37.642352652s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180423099 (101h56m37.642515582s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180455370 (101h56m37.642547853s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180565359 (101h56m37.642657842s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:37784->77.90.31.15:4697: use of closed network connection; conn closed: op Sep 27 19:27:52.180610105 (101h56m37.642702588s) [seg_pos={0 0}, upload=89705]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 27 19:27:52.433487771 (101h56m37.895580254s) [upload=89705]: Done uploading segments... Sep 27 19:27:52.433517006 (101h56m37.89560948s) [upload=89705]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.127532975s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [................................X....X..X.X.................] duration: 6.211195769s FAIL (2025-09-27 19:41:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 27 19:27:22.886943307 (101h56m8.34903578s) [upload=89705]: Starting upload Sep 27 19:27:22.887049439 (101h56m8.349141922s) [upload=89705]: Done starting upload Sep 27 19:27:22.887160700 (101h56m8.349253184s) [upload=89705]: Uploading segments... Sep 27 19:27:22.887344050 (101h56m8.349436533s) [upload=89705]: Got next segment. Inline: false Sep 27 19:27:22.887393123 (101h56m8.349485606s) [upload=89705]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:27:22.887400948 (101h56m8.349493431s) [upload=89705]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:27:23.144152141 (101h56m8.606244624s) [seg_pos={0 0}, upload=89705]: Begin upload segment... Sep 27 19:27:23.144224288 (101h56m8.606316771s) [seg_pos={0 0}, upload=89705]: Done begin upload segment. Sep 27 19:27:23.144303519 (101h56m8.606396002s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144356600 (101h56m8.606449083s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144899694 (101h56m8.606992177s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145386381 (101h56m8.607478864s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145749413 (101h56m8.607841896s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146190593 (101h56m8.608283076s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146569114 (101h56m8.608661597s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146975998 (101h56m8.609068481s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.147437558 (101h56m8.609530051s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.148991526 (101h56m8.611083999s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.151370226 (101h56m8.613462699s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.153185462 (101h56m8.615277966s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.154735545 (101h56m8.616828028s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.158605123 (101h56m8.620697606s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162754133 (101h56m8.624846616s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162908967 (101h56m8.62500145s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:24.838034007 (101h56m10.30012649s) [upload=89705]: Next returned nil segment Sep 27 19:27:24.838098610 (101h56m10.300191093s) [upload=89705]: Waiting for error group managing segments... Sep 27 19:27:25.171482756 (101h56m10.633575229s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.176947972 (101h56m10.639040456s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.180229350 (101h56m10.642321834s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.181707686 (101h56m10.643800159s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.182762434 (101h56m10.644854917s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.192828732 (101h56m10.654921205s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.198888852 (101h56m10.660981335s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.205614519 (101h56m10.667707012s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.207989521 (101h56m10.670081994s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.222032181 (101h56m10.684124654s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.290677625 (101h56m10.752770118s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.652827506 (101h56m11.114919989s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:52.180260159 (101h56m37.642352652s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180423099 (101h56m37.642515582s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180455370 (101h56m37.642547853s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180565359 (101h56m37.642657842s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:37784->77.90.31.15:4697: use of closed network connection; conn closed: op Sep 27 19:27:52.180610105 (101h56m37.642702588s) [seg_pos={0 0}, upload=89705]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 27 19:27:52.433487771 (101h56m37.895580254s) [upload=89705]: Done uploading segments... Sep 27 19:27:52.433517006 (101h56m37.89560948s) [upload=89705]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.211231077s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................................X....X..X.X................] duration: 6.495733464s FAIL (2025-09-27 19:40:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 27 19:27:22.886943307 (101h56m8.34903578s) [upload=89705]: Starting upload Sep 27 19:27:22.887049439 (101h56m8.349141922s) [upload=89705]: Done starting upload Sep 27 19:27:22.887160700 (101h56m8.349253184s) [upload=89705]: Uploading segments... Sep 27 19:27:22.887344050 (101h56m8.349436533s) [upload=89705]: Got next segment. Inline: false Sep 27 19:27:22.887393123 (101h56m8.349485606s) [upload=89705]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:27:22.887400948 (101h56m8.349493431s) [upload=89705]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:27:23.144152141 (101h56m8.606244624s) [seg_pos={0 0}, upload=89705]: Begin upload segment... Sep 27 19:27:23.144224288 (101h56m8.606316771s) [seg_pos={0 0}, upload=89705]: Done begin upload segment. Sep 27 19:27:23.144303519 (101h56m8.606396002s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144356600 (101h56m8.606449083s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144899694 (101h56m8.606992177s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145386381 (101h56m8.607478864s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145749413 (101h56m8.607841896s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146190593 (101h56m8.608283076s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146569114 (101h56m8.608661597s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146975998 (101h56m8.609068481s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.147437558 (101h56m8.609530051s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.148991526 (101h56m8.611083999s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.151370226 (101h56m8.613462699s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.153185462 (101h56m8.615277966s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.154735545 (101h56m8.616828028s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.158605123 (101h56m8.620697606s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162754133 (101h56m8.624846616s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162908967 (101h56m8.62500145s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:24.838034007 (101h56m10.30012649s) [upload=89705]: Next returned nil segment Sep 27 19:27:24.838098610 (101h56m10.300191093s) [upload=89705]: Waiting for error group managing segments... Sep 27 19:27:25.171482756 (101h56m10.633575229s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.176947972 (101h56m10.639040456s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.180229350 (101h56m10.642321834s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.181707686 (101h56m10.643800159s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.182762434 (101h56m10.644854917s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.192828732 (101h56m10.654921205s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.198888852 (101h56m10.660981335s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.205614519 (101h56m10.667707012s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.207989521 (101h56m10.670081994s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.222032181 (101h56m10.684124654s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.290677625 (101h56m10.752770118s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.652827506 (101h56m11.114919989s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:52.180260159 (101h56m37.642352652s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180423099 (101h56m37.642515582s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180455370 (101h56m37.642547853s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180565359 (101h56m37.642657842s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:37784->77.90.31.15:4697: use of closed network connection; conn closed: op Sep 27 19:27:52.180610105 (101h56m37.642702588s) [seg_pos={0 0}, upload=89705]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 27 19:27:52.433487771 (101h56m37.895580254s) [upload=89705]: Done uploading segments... Sep 27 19:27:52.433517006 (101h56m37.89560948s) [upload=89705]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.495765995s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................................X....X..X.X...............] duration: 6.062793563s FAIL (2025-09-27 19:38:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 27 19:27:22.886943307 (101h56m8.34903578s) [upload=89705]: Starting upload Sep 27 19:27:22.887049439 (101h56m8.349141922s) [upload=89705]: Done starting upload Sep 27 19:27:22.887160700 (101h56m8.349253184s) [upload=89705]: Uploading segments... Sep 27 19:27:22.887344050 (101h56m8.349436533s) [upload=89705]: Got next segment. Inline: false Sep 27 19:27:22.887393123 (101h56m8.349485606s) [upload=89705]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:27:22.887400948 (101h56m8.349493431s) [upload=89705]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:27:23.144152141 (101h56m8.606244624s) [seg_pos={0 0}, upload=89705]: Begin upload segment... Sep 27 19:27:23.144224288 (101h56m8.606316771s) [seg_pos={0 0}, upload=89705]: Done begin upload segment. Sep 27 19:27:23.144303519 (101h56m8.606396002s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144356600 (101h56m8.606449083s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144899694 (101h56m8.606992177s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145386381 (101h56m8.607478864s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145749413 (101h56m8.607841896s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146190593 (101h56m8.608283076s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146569114 (101h56m8.608661597s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146975998 (101h56m8.609068481s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.147437558 (101h56m8.609530051s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.148991526 (101h56m8.611083999s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.151370226 (101h56m8.613462699s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.153185462 (101h56m8.615277966s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.154735545 (101h56m8.616828028s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.158605123 (101h56m8.620697606s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162754133 (101h56m8.624846616s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162908967 (101h56m8.62500145s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:24.838034007 (101h56m10.30012649s) [upload=89705]: Next returned nil segment Sep 27 19:27:24.838098610 (101h56m10.300191093s) [upload=89705]: Waiting for error group managing segments... Sep 27 19:27:25.171482756 (101h56m10.633575229s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.176947972 (101h56m10.639040456s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.180229350 (101h56m10.642321834s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.181707686 (101h56m10.643800159s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.182762434 (101h56m10.644854917s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.192828732 (101h56m10.654921205s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.198888852 (101h56m10.660981335s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.205614519 (101h56m10.667707012s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.207989521 (101h56m10.670081994s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.222032181 (101h56m10.684124654s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.290677625 (101h56m10.752770118s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.652827506 (101h56m11.114919989s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:52.180260159 (101h56m37.642352652s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180423099 (101h56m37.642515582s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180455370 (101h56m37.642547853s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180565359 (101h56m37.642657842s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:37784->77.90.31.15:4697: use of closed network connection; conn closed: op Sep 27 19:27:52.180610105 (101h56m37.642702588s) [seg_pos={0 0}, upload=89705]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 27 19:27:52.433487771 (101h56m37.895580254s) [upload=89705]: Done uploading segments... Sep 27 19:27:52.433517006 (101h56m37.89560948s) [upload=89705]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.062826225s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................................X....X..X.X..............] duration: 6.88150586s FAIL (2025-09-27 19:38:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 27 19:27:22.886943307 (101h56m8.34903578s) [upload=89705]: Starting upload Sep 27 19:27:22.887049439 (101h56m8.349141922s) [upload=89705]: Done starting upload Sep 27 19:27:22.887160700 (101h56m8.349253184s) [upload=89705]: Uploading segments... Sep 27 19:27:22.887344050 (101h56m8.349436533s) [upload=89705]: Got next segment. Inline: false Sep 27 19:27:22.887393123 (101h56m8.349485606s) [upload=89705]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:27:22.887400948 (101h56m8.349493431s) [upload=89705]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:27:23.144152141 (101h56m8.606244624s) [seg_pos={0 0}, upload=89705]: Begin upload segment... Sep 27 19:27:23.144224288 (101h56m8.606316771s) [seg_pos={0 0}, upload=89705]: Done begin upload segment. Sep 27 19:27:23.144303519 (101h56m8.606396002s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144356600 (101h56m8.606449083s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144899694 (101h56m8.606992177s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145386381 (101h56m8.607478864s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145749413 (101h56m8.607841896s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146190593 (101h56m8.608283076s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146569114 (101h56m8.608661597s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146975998 (101h56m8.609068481s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.147437558 (101h56m8.609530051s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.148991526 (101h56m8.611083999s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.151370226 (101h56m8.613462699s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.153185462 (101h56m8.615277966s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.154735545 (101h56m8.616828028s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.158605123 (101h56m8.620697606s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162754133 (101h56m8.624846616s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162908967 (101h56m8.62500145s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:24.838034007 (101h56m10.30012649s) [upload=89705]: Next returned nil segment Sep 27 19:27:24.838098610 (101h56m10.300191093s) [upload=89705]: Waiting for error group managing segments... Sep 27 19:27:25.171482756 (101h56m10.633575229s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.176947972 (101h56m10.639040456s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.180229350 (101h56m10.642321834s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.181707686 (101h56m10.643800159s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.182762434 (101h56m10.644854917s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.192828732 (101h56m10.654921205s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.198888852 (101h56m10.660981335s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.205614519 (101h56m10.667707012s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.207989521 (101h56m10.670081994s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.222032181 (101h56m10.684124654s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.290677625 (101h56m10.752770118s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.652827506 (101h56m11.114919989s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:52.180260159 (101h56m37.642352652s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180423099 (101h56m37.642515582s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180455370 (101h56m37.642547853s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180565359 (101h56m37.642657842s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:37784->77.90.31.15:4697: use of closed network connection; conn closed: op Sep 27 19:27:52.180610105 (101h56m37.642702588s) [seg_pos={0 0}, upload=89705]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 27 19:27:52.433487771 (101h56m37.895580254s) [upload=89705]: Done uploading segments... Sep 27 19:27:52.433517006 (101h56m37.89560948s) [upload=89705]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.881545977s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................................X....X..X.X.............] duration: 24.209417577s FAIL (2025-09-27 19:37:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 27 19:27:22.886943307 (101h56m8.34903578s) [upload=89705]: Starting upload Sep 27 19:27:22.887049439 (101h56m8.349141922s) [upload=89705]: Done starting upload Sep 27 19:27:22.887160700 (101h56m8.349253184s) [upload=89705]: Uploading segments... Sep 27 19:27:22.887344050 (101h56m8.349436533s) [upload=89705]: Got next segment. Inline: false Sep 27 19:27:22.887393123 (101h56m8.349485606s) [upload=89705]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:27:22.887400948 (101h56m8.349493431s) [upload=89705]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:27:23.144152141 (101h56m8.606244624s) [seg_pos={0 0}, upload=89705]: Begin upload segment... Sep 27 19:27:23.144224288 (101h56m8.606316771s) [seg_pos={0 0}, upload=89705]: Done begin upload segment. Sep 27 19:27:23.144303519 (101h56m8.606396002s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144356600 (101h56m8.606449083s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144899694 (101h56m8.606992177s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145386381 (101h56m8.607478864s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145749413 (101h56m8.607841896s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146190593 (101h56m8.608283076s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146569114 (101h56m8.608661597s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146975998 (101h56m8.609068481s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.147437558 (101h56m8.609530051s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.148991526 (101h56m8.611083999s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.151370226 (101h56m8.613462699s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.153185462 (101h56m8.615277966s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.154735545 (101h56m8.616828028s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.158605123 (101h56m8.620697606s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162754133 (101h56m8.624846616s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162908967 (101h56m8.62500145s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:24.838034007 (101h56m10.30012649s) [upload=89705]: Next returned nil segment Sep 27 19:27:24.838098610 (101h56m10.300191093s) [upload=89705]: Waiting for error group managing segments... Sep 27 19:27:25.171482756 (101h56m10.633575229s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.176947972 (101h56m10.639040456s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.180229350 (101h56m10.642321834s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.181707686 (101h56m10.643800159s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.182762434 (101h56m10.644854917s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.192828732 (101h56m10.654921205s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.198888852 (101h56m10.660981335s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.205614519 (101h56m10.667707012s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.207989521 (101h56m10.670081994s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.222032181 (101h56m10.684124654s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.290677625 (101h56m10.752770118s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.652827506 (101h56m11.114919989s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:52.180260159 (101h56m37.642352652s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180423099 (101h56m37.642515582s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180455370 (101h56m37.642547853s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180565359 (101h56m37.642657842s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:37784->77.90.31.15:4697: use of closed network connection; conn closed: op Sep 27 19:27:52.180610105 (101h56m37.642702588s) [seg_pos={0 0}, upload=89705]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 27 19:27:52.433487771 (101h56m37.895580254s) [upload=89705]: Done uploading segments... Sep 27 19:27:52.433517006 (101h56m37.89560948s) [upload=89705]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 24.209446733s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................................X....X..X.X............] duration: 22.067376317s FAIL (2025-09-27 19:36:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 27 19:27:22.886943307 (101h56m8.34903578s) [upload=89705]: Starting upload Sep 27 19:27:22.887049439 (101h56m8.349141922s) [upload=89705]: Done starting upload Sep 27 19:27:22.887160700 (101h56m8.349253184s) [upload=89705]: Uploading segments... Sep 27 19:27:22.887344050 (101h56m8.349436533s) [upload=89705]: Got next segment. Inline: false Sep 27 19:27:22.887393123 (101h56m8.349485606s) [upload=89705]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:27:22.887400948 (101h56m8.349493431s) [upload=89705]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:27:23.144152141 (101h56m8.606244624s) [seg_pos={0 0}, upload=89705]: Begin upload segment... Sep 27 19:27:23.144224288 (101h56m8.606316771s) [seg_pos={0 0}, upload=89705]: Done begin upload segment. Sep 27 19:27:23.144303519 (101h56m8.606396002s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144356600 (101h56m8.606449083s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144899694 (101h56m8.606992177s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145386381 (101h56m8.607478864s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145749413 (101h56m8.607841896s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146190593 (101h56m8.608283076s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146569114 (101h56m8.608661597s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146975998 (101h56m8.609068481s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.147437558 (101h56m8.609530051s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.148991526 (101h56m8.611083999s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.151370226 (101h56m8.613462699s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.153185462 (101h56m8.615277966s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.154735545 (101h56m8.616828028s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.158605123 (101h56m8.620697606s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162754133 (101h56m8.624846616s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162908967 (101h56m8.62500145s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:24.838034007 (101h56m10.30012649s) [upload=89705]: Next returned nil segment Sep 27 19:27:24.838098610 (101h56m10.300191093s) [upload=89705]: Waiting for error group managing segments... Sep 27 19:27:25.171482756 (101h56m10.633575229s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.176947972 (101h56m10.639040456s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.180229350 (101h56m10.642321834s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.181707686 (101h56m10.643800159s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.182762434 (101h56m10.644854917s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.192828732 (101h56m10.654921205s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.198888852 (101h56m10.660981335s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.205614519 (101h56m10.667707012s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.207989521 (101h56m10.670081994s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.222032181 (101h56m10.684124654s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.290677625 (101h56m10.752770118s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.652827506 (101h56m11.114919989s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:52.180260159 (101h56m37.642352652s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180423099 (101h56m37.642515582s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180455370 (101h56m37.642547853s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180565359 (101h56m37.642657842s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:37784->77.90.31.15:4697: use of closed network connection; conn closed: op Sep 27 19:27:52.180610105 (101h56m37.642702588s) [seg_pos={0 0}, upload=89705]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 27 19:27:52.433487771 (101h56m37.895580254s) [upload=89705]: Done uploading segments... Sep 27 19:27:52.433517006 (101h56m37.89560948s) [upload=89705]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 22.067489854s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................................X....X..X.X...........] duration: 6.643614694s FAIL (2025-09-27 19:35:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 27 19:27:22.886943307 (101h56m8.34903578s) [upload=89705]: Starting upload Sep 27 19:27:22.887049439 (101h56m8.349141922s) [upload=89705]: Done starting upload Sep 27 19:27:22.887160700 (101h56m8.349253184s) [upload=89705]: Uploading segments... Sep 27 19:27:22.887344050 (101h56m8.349436533s) [upload=89705]: Got next segment. Inline: false Sep 27 19:27:22.887393123 (101h56m8.349485606s) [upload=89705]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:27:22.887400948 (101h56m8.349493431s) [upload=89705]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:27:23.144152141 (101h56m8.606244624s) [seg_pos={0 0}, upload=89705]: Begin upload segment... Sep 27 19:27:23.144224288 (101h56m8.606316771s) [seg_pos={0 0}, upload=89705]: Done begin upload segment. Sep 27 19:27:23.144303519 (101h56m8.606396002s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144356600 (101h56m8.606449083s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144899694 (101h56m8.606992177s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145386381 (101h56m8.607478864s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145749413 (101h56m8.607841896s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146190593 (101h56m8.608283076s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146569114 (101h56m8.608661597s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146975998 (101h56m8.609068481s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.147437558 (101h56m8.609530051s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.148991526 (101h56m8.611083999s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.151370226 (101h56m8.613462699s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.153185462 (101h56m8.615277966s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.154735545 (101h56m8.616828028s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.158605123 (101h56m8.620697606s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162754133 (101h56m8.624846616s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162908967 (101h56m8.62500145s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:24.838034007 (101h56m10.30012649s) [upload=89705]: Next returned nil segment Sep 27 19:27:24.838098610 (101h56m10.300191093s) [upload=89705]: Waiting for error group managing segments... Sep 27 19:27:25.171482756 (101h56m10.633575229s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.176947972 (101h56m10.639040456s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.180229350 (101h56m10.642321834s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.181707686 (101h56m10.643800159s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.182762434 (101h56m10.644854917s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.192828732 (101h56m10.654921205s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.198888852 (101h56m10.660981335s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.205614519 (101h56m10.667707012s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.207989521 (101h56m10.670081994s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.222032181 (101h56m10.684124654s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.290677625 (101h56m10.752770118s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.652827506 (101h56m11.114919989s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:52.180260159 (101h56m37.642352652s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180423099 (101h56m37.642515582s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180455370 (101h56m37.642547853s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180565359 (101h56m37.642657842s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:37784->77.90.31.15:4697: use of closed network connection; conn closed: op Sep 27 19:27:52.180610105 (101h56m37.642702588s) [seg_pos={0 0}, upload=89705]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 27 19:27:52.433487771 (101h56m37.895580254s) [upload=89705]: Done uploading segments... Sep 27 19:27:52.433517006 (101h56m37.89560948s) [upload=89705]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.643645523s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................................X....X..X.X..........] duration: 5.919015369s FAIL (2025-09-27 19:35:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 27 19:27:22.886943307 (101h56m8.34903578s) [upload=89705]: Starting upload Sep 27 19:27:22.887049439 (101h56m8.349141922s) [upload=89705]: Done starting upload Sep 27 19:27:22.887160700 (101h56m8.349253184s) [upload=89705]: Uploading segments... Sep 27 19:27:22.887344050 (101h56m8.349436533s) [upload=89705]: Got next segment. Inline: false Sep 27 19:27:22.887393123 (101h56m8.349485606s) [upload=89705]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:27:22.887400948 (101h56m8.349493431s) [upload=89705]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:27:23.144152141 (101h56m8.606244624s) [seg_pos={0 0}, upload=89705]: Begin upload segment... Sep 27 19:27:23.144224288 (101h56m8.606316771s) [seg_pos={0 0}, upload=89705]: Done begin upload segment. Sep 27 19:27:23.144303519 (101h56m8.606396002s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144356600 (101h56m8.606449083s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144899694 (101h56m8.606992177s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145386381 (101h56m8.607478864s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145749413 (101h56m8.607841896s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146190593 (101h56m8.608283076s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146569114 (101h56m8.608661597s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146975998 (101h56m8.609068481s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.147437558 (101h56m8.609530051s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.148991526 (101h56m8.611083999s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.151370226 (101h56m8.613462699s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.153185462 (101h56m8.615277966s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.154735545 (101h56m8.616828028s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.158605123 (101h56m8.620697606s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162754133 (101h56m8.624846616s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162908967 (101h56m8.62500145s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:24.838034007 (101h56m10.30012649s) [upload=89705]: Next returned nil segment Sep 27 19:27:24.838098610 (101h56m10.300191093s) [upload=89705]: Waiting for error group managing segments... Sep 27 19:27:25.171482756 (101h56m10.633575229s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.176947972 (101h56m10.639040456s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.180229350 (101h56m10.642321834s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.181707686 (101h56m10.643800159s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.182762434 (101h56m10.644854917s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.192828732 (101h56m10.654921205s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.198888852 (101h56m10.660981335s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.205614519 (101h56m10.667707012s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.207989521 (101h56m10.670081994s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.222032181 (101h56m10.684124654s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.290677625 (101h56m10.752770118s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.652827506 (101h56m11.114919989s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:52.180260159 (101h56m37.642352652s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180423099 (101h56m37.642515582s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180455370 (101h56m37.642547853s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180565359 (101h56m37.642657842s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:37784->77.90.31.15:4697: use of closed network connection; conn closed: op Sep 27 19:27:52.180610105 (101h56m37.642702588s) [seg_pos={0 0}, upload=89705]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 27 19:27:52.433487771 (101h56m37.895580254s) [upload=89705]: Done uploading segments... Sep 27 19:27:52.433517006 (101h56m37.89560948s) [upload=89705]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.919042591s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................................X....X..X.X.........] duration: 9.942926797s FAIL (2025-09-27 19:34:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 27 19:27:22.886943307 (101h56m8.34903578s) [upload=89705]: Starting upload Sep 27 19:27:22.887049439 (101h56m8.349141922s) [upload=89705]: Done starting upload Sep 27 19:27:22.887160700 (101h56m8.349253184s) [upload=89705]: Uploading segments... Sep 27 19:27:22.887344050 (101h56m8.349436533s) [upload=89705]: Got next segment. Inline: false Sep 27 19:27:22.887393123 (101h56m8.349485606s) [upload=89705]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:27:22.887400948 (101h56m8.349493431s) [upload=89705]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:27:23.144152141 (101h56m8.606244624s) [seg_pos={0 0}, upload=89705]: Begin upload segment... Sep 27 19:27:23.144224288 (101h56m8.606316771s) [seg_pos={0 0}, upload=89705]: Done begin upload segment. Sep 27 19:27:23.144303519 (101h56m8.606396002s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144356600 (101h56m8.606449083s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144899694 (101h56m8.606992177s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145386381 (101h56m8.607478864s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145749413 (101h56m8.607841896s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146190593 (101h56m8.608283076s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146569114 (101h56m8.608661597s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146975998 (101h56m8.609068481s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.147437558 (101h56m8.609530051s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.148991526 (101h56m8.611083999s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.151370226 (101h56m8.613462699s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.153185462 (101h56m8.615277966s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.154735545 (101h56m8.616828028s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.158605123 (101h56m8.620697606s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162754133 (101h56m8.624846616s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162908967 (101h56m8.62500145s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:24.838034007 (101h56m10.30012649s) [upload=89705]: Next returned nil segment Sep 27 19:27:24.838098610 (101h56m10.300191093s) [upload=89705]: Waiting for error group managing segments... Sep 27 19:27:25.171482756 (101h56m10.633575229s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.176947972 (101h56m10.639040456s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.180229350 (101h56m10.642321834s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.181707686 (101h56m10.643800159s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.182762434 (101h56m10.644854917s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.192828732 (101h56m10.654921205s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.198888852 (101h56m10.660981335s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.205614519 (101h56m10.667707012s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.207989521 (101h56m10.670081994s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.222032181 (101h56m10.684124654s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.290677625 (101h56m10.752770118s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.652827506 (101h56m11.114919989s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:52.180260159 (101h56m37.642352652s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180423099 (101h56m37.642515582s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180455370 (101h56m37.642547853s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180565359 (101h56m37.642657842s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:37784->77.90.31.15:4697: use of closed network connection; conn closed: op Sep 27 19:27:52.180610105 (101h56m37.642702588s) [seg_pos={0 0}, upload=89705]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 27 19:27:52.433487771 (101h56m37.895580254s) [upload=89705]: Done uploading segments... Sep 27 19:27:52.433517006 (101h56m37.89560948s) [upload=89705]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 9.942952345s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........................................X....X..X.X........] duration: 9.536787326s FAIL (2025-09-27 19:33:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 27 19:27:22.886943307 (101h56m8.34903578s) [upload=89705]: Starting upload Sep 27 19:27:22.887049439 (101h56m8.349141922s) [upload=89705]: Done starting upload Sep 27 19:27:22.887160700 (101h56m8.349253184s) [upload=89705]: Uploading segments... Sep 27 19:27:22.887344050 (101h56m8.349436533s) [upload=89705]: Got next segment. Inline: false Sep 27 19:27:22.887393123 (101h56m8.349485606s) [upload=89705]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:27:22.887400948 (101h56m8.349493431s) [upload=89705]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:27:23.144152141 (101h56m8.606244624s) [seg_pos={0 0}, upload=89705]: Begin upload segment... Sep 27 19:27:23.144224288 (101h56m8.606316771s) [seg_pos={0 0}, upload=89705]: Done begin upload segment. Sep 27 19:27:23.144303519 (101h56m8.606396002s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144356600 (101h56m8.606449083s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144899694 (101h56m8.606992177s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145386381 (101h56m8.607478864s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145749413 (101h56m8.607841896s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146190593 (101h56m8.608283076s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146569114 (101h56m8.608661597s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146975998 (101h56m8.609068481s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.147437558 (101h56m8.609530051s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.148991526 (101h56m8.611083999s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.151370226 (101h56m8.613462699s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.153185462 (101h56m8.615277966s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.154735545 (101h56m8.616828028s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.158605123 (101h56m8.620697606s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162754133 (101h56m8.624846616s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162908967 (101h56m8.62500145s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:24.838034007 (101h56m10.30012649s) [upload=89705]: Next returned nil segment Sep 27 19:27:24.838098610 (101h56m10.300191093s) [upload=89705]: Waiting for error group managing segments... Sep 27 19:27:25.171482756 (101h56m10.633575229s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.176947972 (101h56m10.639040456s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.180229350 (101h56m10.642321834s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.181707686 (101h56m10.643800159s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.182762434 (101h56m10.644854917s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.192828732 (101h56m10.654921205s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.198888852 (101h56m10.660981335s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.205614519 (101h56m10.667707012s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.207989521 (101h56m10.670081994s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.222032181 (101h56m10.684124654s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.290677625 (101h56m10.752770118s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.652827506 (101h56m11.114919989s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:52.180260159 (101h56m37.642352652s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180423099 (101h56m37.642515582s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180455370 (101h56m37.642547853s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180565359 (101h56m37.642657842s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:37784->77.90.31.15:4697: use of closed network connection; conn closed: op Sep 27 19:27:52.180610105 (101h56m37.642702588s) [seg_pos={0 0}, upload=89705]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 27 19:27:52.433487771 (101h56m37.895580254s) [upload=89705]: Done uploading segments... Sep 27 19:27:52.433517006 (101h56m37.89560948s) [upload=89705]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 9.536808887s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........................................X....X..X.X.......] duration: 6.176870054s FAIL (2025-09-27 19:32:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 27 19:27:22.886943307 (101h56m8.34903578s) [upload=89705]: Starting upload Sep 27 19:27:22.887049439 (101h56m8.349141922s) [upload=89705]: Done starting upload Sep 27 19:27:22.887160700 (101h56m8.349253184s) [upload=89705]: Uploading segments... Sep 27 19:27:22.887344050 (101h56m8.349436533s) [upload=89705]: Got next segment. Inline: false Sep 27 19:27:22.887393123 (101h56m8.349485606s) [upload=89705]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:27:22.887400948 (101h56m8.349493431s) [upload=89705]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:27:23.144152141 (101h56m8.606244624s) [seg_pos={0 0}, upload=89705]: Begin upload segment... Sep 27 19:27:23.144224288 (101h56m8.606316771s) [seg_pos={0 0}, upload=89705]: Done begin upload segment. Sep 27 19:27:23.144303519 (101h56m8.606396002s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144356600 (101h56m8.606449083s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144899694 (101h56m8.606992177s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145386381 (101h56m8.607478864s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145749413 (101h56m8.607841896s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146190593 (101h56m8.608283076s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146569114 (101h56m8.608661597s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146975998 (101h56m8.609068481s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.147437558 (101h56m8.609530051s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.148991526 (101h56m8.611083999s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.151370226 (101h56m8.613462699s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.153185462 (101h56m8.615277966s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.154735545 (101h56m8.616828028s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.158605123 (101h56m8.620697606s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162754133 (101h56m8.624846616s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162908967 (101h56m8.62500145s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:24.838034007 (101h56m10.30012649s) [upload=89705]: Next returned nil segment Sep 27 19:27:24.838098610 (101h56m10.300191093s) [upload=89705]: Waiting for error group managing segments... Sep 27 19:27:25.171482756 (101h56m10.633575229s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.176947972 (101h56m10.639040456s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.180229350 (101h56m10.642321834s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.181707686 (101h56m10.643800159s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.182762434 (101h56m10.644854917s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.192828732 (101h56m10.654921205s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.198888852 (101h56m10.660981335s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.205614519 (101h56m10.667707012s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.207989521 (101h56m10.670081994s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.222032181 (101h56m10.684124654s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.290677625 (101h56m10.752770118s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.652827506 (101h56m11.114919989s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:52.180260159 (101h56m37.642352652s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180423099 (101h56m37.642515582s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180455370 (101h56m37.642547853s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180565359 (101h56m37.642657842s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:37784->77.90.31.15:4697: use of closed network connection; conn closed: op Sep 27 19:27:52.180610105 (101h56m37.642702588s) [seg_pos={0 0}, upload=89705]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 27 19:27:52.433487771 (101h56m37.895580254s) [upload=89705]: Done uploading segments... Sep 27 19:27:52.433517006 (101h56m37.89560948s) [upload=89705]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.176908197s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........................................X....X..X.X......] duration: 6.422584994s FAIL (2025-09-27 19:31:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 27 19:27:22.886943307 (101h56m8.34903578s) [upload=89705]: Starting upload Sep 27 19:27:22.887049439 (101h56m8.349141922s) [upload=89705]: Done starting upload Sep 27 19:27:22.887160700 (101h56m8.349253184s) [upload=89705]: Uploading segments... Sep 27 19:27:22.887344050 (101h56m8.349436533s) [upload=89705]: Got next segment. Inline: false Sep 27 19:27:22.887393123 (101h56m8.349485606s) [upload=89705]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:27:22.887400948 (101h56m8.349493431s) [upload=89705]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:27:23.144152141 (101h56m8.606244624s) [seg_pos={0 0}, upload=89705]: Begin upload segment... Sep 27 19:27:23.144224288 (101h56m8.606316771s) [seg_pos={0 0}, upload=89705]: Done begin upload segment. Sep 27 19:27:23.144303519 (101h56m8.606396002s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144356600 (101h56m8.606449083s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144899694 (101h56m8.606992177s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145386381 (101h56m8.607478864s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145749413 (101h56m8.607841896s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146190593 (101h56m8.608283076s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146569114 (101h56m8.608661597s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146975998 (101h56m8.609068481s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.147437558 (101h56m8.609530051s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.148991526 (101h56m8.611083999s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.151370226 (101h56m8.613462699s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.153185462 (101h56m8.615277966s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.154735545 (101h56m8.616828028s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.158605123 (101h56m8.620697606s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162754133 (101h56m8.624846616s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162908967 (101h56m8.62500145s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:24.838034007 (101h56m10.30012649s) [upload=89705]: Next returned nil segment Sep 27 19:27:24.838098610 (101h56m10.300191093s) [upload=89705]: Waiting for error group managing segments... Sep 27 19:27:25.171482756 (101h56m10.633575229s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.176947972 (101h56m10.639040456s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.180229350 (101h56m10.642321834s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.181707686 (101h56m10.643800159s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.182762434 (101h56m10.644854917s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.192828732 (101h56m10.654921205s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.198888852 (101h56m10.660981335s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.205614519 (101h56m10.667707012s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.207989521 (101h56m10.670081994s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.222032181 (101h56m10.684124654s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.290677625 (101h56m10.752770118s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.652827506 (101h56m11.114919989s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:52.180260159 (101h56m37.642352652s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180423099 (101h56m37.642515582s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180455370 (101h56m37.642547853s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180565359 (101h56m37.642657842s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:37784->77.90.31.15:4697: use of closed network connection; conn closed: op Sep 27 19:27:52.180610105 (101h56m37.642702588s) [seg_pos={0 0}, upload=89705]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 27 19:27:52.433487771 (101h56m37.895580254s) [upload=89705]: Done uploading segments... Sep 27 19:27:52.433517006 (101h56m37.89560948s) [upload=89705]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.422635721s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [............................................X....X..X.X.....] duration: 6.189543084s FAIL (2025-09-27 19:30:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 27 19:27:22.886943307 (101h56m8.34903578s) [upload=89705]: Starting upload Sep 27 19:27:22.887049439 (101h56m8.349141922s) [upload=89705]: Done starting upload Sep 27 19:27:22.887160700 (101h56m8.349253184s) [upload=89705]: Uploading segments... Sep 27 19:27:22.887344050 (101h56m8.349436533s) [upload=89705]: Got next segment. Inline: false Sep 27 19:27:22.887393123 (101h56m8.349485606s) [upload=89705]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:27:22.887400948 (101h56m8.349493431s) [upload=89705]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:27:23.144152141 (101h56m8.606244624s) [seg_pos={0 0}, upload=89705]: Begin upload segment... Sep 27 19:27:23.144224288 (101h56m8.606316771s) [seg_pos={0 0}, upload=89705]: Done begin upload segment. Sep 27 19:27:23.144303519 (101h56m8.606396002s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144356600 (101h56m8.606449083s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144899694 (101h56m8.606992177s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145386381 (101h56m8.607478864s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145749413 (101h56m8.607841896s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146190593 (101h56m8.608283076s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146569114 (101h56m8.608661597s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146975998 (101h56m8.609068481s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.147437558 (101h56m8.609530051s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.148991526 (101h56m8.611083999s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.151370226 (101h56m8.613462699s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.153185462 (101h56m8.615277966s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.154735545 (101h56m8.616828028s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.158605123 (101h56m8.620697606s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162754133 (101h56m8.624846616s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162908967 (101h56m8.62500145s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:24.838034007 (101h56m10.30012649s) [upload=89705]: Next returned nil segment Sep 27 19:27:24.838098610 (101h56m10.300191093s) [upload=89705]: Waiting for error group managing segments... Sep 27 19:27:25.171482756 (101h56m10.633575229s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.176947972 (101h56m10.639040456s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.180229350 (101h56m10.642321834s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.181707686 (101h56m10.643800159s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.182762434 (101h56m10.644854917s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.192828732 (101h56m10.654921205s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.198888852 (101h56m10.660981335s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.205614519 (101h56m10.667707012s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.207989521 (101h56m10.670081994s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.222032181 (101h56m10.684124654s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.290677625 (101h56m10.752770118s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.652827506 (101h56m11.114919989s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:52.180260159 (101h56m37.642352652s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180423099 (101h56m37.642515582s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180455370 (101h56m37.642547853s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180565359 (101h56m37.642657842s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:37784->77.90.31.15:4697: use of closed network connection; conn closed: op Sep 27 19:27:52.180610105 (101h56m37.642702588s) [seg_pos={0 0}, upload=89705]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 27 19:27:52.433487771 (101h56m37.895580254s) [upload=89705]: Done uploading segments... Sep 27 19:27:52.433517006 (101h56m37.89560948s) [upload=89705]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.189577207s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............................................X....X..X.X....] duration: 6.215263042s FAIL (2025-09-27 19:29:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 27 19:27:22.886943307 (101h56m8.34903578s) [upload=89705]: Starting upload Sep 27 19:27:22.887049439 (101h56m8.349141922s) [upload=89705]: Done starting upload Sep 27 19:27:22.887160700 (101h56m8.349253184s) [upload=89705]: Uploading segments... Sep 27 19:27:22.887344050 (101h56m8.349436533s) [upload=89705]: Got next segment. Inline: false Sep 27 19:27:22.887393123 (101h56m8.349485606s) [upload=89705]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:27:22.887400948 (101h56m8.349493431s) [upload=89705]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:27:23.144152141 (101h56m8.606244624s) [seg_pos={0 0}, upload=89705]: Begin upload segment... Sep 27 19:27:23.144224288 (101h56m8.606316771s) [seg_pos={0 0}, upload=89705]: Done begin upload segment. Sep 27 19:27:23.144303519 (101h56m8.606396002s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144356600 (101h56m8.606449083s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144899694 (101h56m8.606992177s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145386381 (101h56m8.607478864s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145749413 (101h56m8.607841896s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146190593 (101h56m8.608283076s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146569114 (101h56m8.608661597s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146975998 (101h56m8.609068481s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.147437558 (101h56m8.609530051s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.148991526 (101h56m8.611083999s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.151370226 (101h56m8.613462699s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.153185462 (101h56m8.615277966s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.154735545 (101h56m8.616828028s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.158605123 (101h56m8.620697606s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162754133 (101h56m8.624846616s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162908967 (101h56m8.62500145s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:24.838034007 (101h56m10.30012649s) [upload=89705]: Next returned nil segment Sep 27 19:27:24.838098610 (101h56m10.300191093s) [upload=89705]: Waiting for error group managing segments... Sep 27 19:27:25.171482756 (101h56m10.633575229s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.176947972 (101h56m10.639040456s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.180229350 (101h56m10.642321834s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.181707686 (101h56m10.643800159s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.182762434 (101h56m10.644854917s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.192828732 (101h56m10.654921205s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.198888852 (101h56m10.660981335s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.205614519 (101h56m10.667707012s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.207989521 (101h56m10.670081994s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.222032181 (101h56m10.684124654s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.290677625 (101h56m10.752770118s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.652827506 (101h56m11.114919989s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:52.180260159 (101h56m37.642352652s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180423099 (101h56m37.642515582s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180455370 (101h56m37.642547853s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180565359 (101h56m37.642657842s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:37784->77.90.31.15:4697: use of closed network connection; conn closed: op Sep 27 19:27:52.180610105 (101h56m37.642702588s) [seg_pos={0 0}, upload=89705]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 27 19:27:52.433487771 (101h56m37.895580254s) [upload=89705]: Done uploading segments... Sep 27 19:27:52.433517006 (101h56m37.89560948s) [upload=89705]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.215291838s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............................................X....X..X.X...] duration: 6.053392037s FAIL (2025-09-27 19:29:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 27 19:27:22.886943307 (101h56m8.34903578s) [upload=89705]: Starting upload Sep 27 19:27:22.887049439 (101h56m8.349141922s) [upload=89705]: Done starting upload Sep 27 19:27:22.887160700 (101h56m8.349253184s) [upload=89705]: Uploading segments... Sep 27 19:27:22.887344050 (101h56m8.349436533s) [upload=89705]: Got next segment. Inline: false Sep 27 19:27:22.887393123 (101h56m8.349485606s) [upload=89705]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:27:22.887400948 (101h56m8.349493431s) [upload=89705]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:27:23.144152141 (101h56m8.606244624s) [seg_pos={0 0}, upload=89705]: Begin upload segment... Sep 27 19:27:23.144224288 (101h56m8.606316771s) [seg_pos={0 0}, upload=89705]: Done begin upload segment. Sep 27 19:27:23.144303519 (101h56m8.606396002s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144356600 (101h56m8.606449083s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144899694 (101h56m8.606992177s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145386381 (101h56m8.607478864s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145749413 (101h56m8.607841896s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146190593 (101h56m8.608283076s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146569114 (101h56m8.608661597s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146975998 (101h56m8.609068481s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.147437558 (101h56m8.609530051s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.148991526 (101h56m8.611083999s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.151370226 (101h56m8.613462699s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.153185462 (101h56m8.615277966s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.154735545 (101h56m8.616828028s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.158605123 (101h56m8.620697606s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162754133 (101h56m8.624846616s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162908967 (101h56m8.62500145s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:24.838034007 (101h56m10.30012649s) [upload=89705]: Next returned nil segment Sep 27 19:27:24.838098610 (101h56m10.300191093s) [upload=89705]: Waiting for error group managing segments... Sep 27 19:27:25.171482756 (101h56m10.633575229s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.176947972 (101h56m10.639040456s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.180229350 (101h56m10.642321834s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.181707686 (101h56m10.643800159s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.182762434 (101h56m10.644854917s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.192828732 (101h56m10.654921205s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.198888852 (101h56m10.660981335s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.205614519 (101h56m10.667707012s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.207989521 (101h56m10.670081994s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.222032181 (101h56m10.684124654s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.290677625 (101h56m10.752770118s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.652827506 (101h56m11.114919989s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:52.180260159 (101h56m37.642352652s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180423099 (101h56m37.642515582s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180455370 (101h56m37.642547853s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180565359 (101h56m37.642657842s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:37784->77.90.31.15:4697: use of closed network connection; conn closed: op Sep 27 19:27:52.180610105 (101h56m37.642702588s) [seg_pos={0 0}, upload=89705]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 27 19:27:52.433487771 (101h56m37.895580254s) [upload=89705]: Done uploading segments... Sep 27 19:27:52.433517006 (101h56m37.89560948s) [upload=89705]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.053418969s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............................................X....X..X.X..] duration: 6.269700678s FAIL (2025-09-27 19:28:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 27 19:27:22.886943307 (101h56m8.34903578s) [upload=89705]: Starting upload Sep 27 19:27:22.887049439 (101h56m8.349141922s) [upload=89705]: Done starting upload Sep 27 19:27:22.887160700 (101h56m8.349253184s) [upload=89705]: Uploading segments... Sep 27 19:27:22.887344050 (101h56m8.349436533s) [upload=89705]: Got next segment. Inline: false Sep 27 19:27:22.887393123 (101h56m8.349485606s) [upload=89705]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:27:22.887400948 (101h56m8.349493431s) [upload=89705]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:27:23.144152141 (101h56m8.606244624s) [seg_pos={0 0}, upload=89705]: Begin upload segment... Sep 27 19:27:23.144224288 (101h56m8.606316771s) [seg_pos={0 0}, upload=89705]: Done begin upload segment. Sep 27 19:27:23.144303519 (101h56m8.606396002s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144356600 (101h56m8.606449083s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144899694 (101h56m8.606992177s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145386381 (101h56m8.607478864s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145749413 (101h56m8.607841896s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146190593 (101h56m8.608283076s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146569114 (101h56m8.608661597s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146975998 (101h56m8.609068481s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.147437558 (101h56m8.609530051s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.148991526 (101h56m8.611083999s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.151370226 (101h56m8.613462699s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.153185462 (101h56m8.615277966s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.154735545 (101h56m8.616828028s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.158605123 (101h56m8.620697606s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162754133 (101h56m8.624846616s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162908967 (101h56m8.62500145s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:24.838034007 (101h56m10.30012649s) [upload=89705]: Next returned nil segment Sep 27 19:27:24.838098610 (101h56m10.300191093s) [upload=89705]: Waiting for error group managing segments... Sep 27 19:27:25.171482756 (101h56m10.633575229s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.176947972 (101h56m10.639040456s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.180229350 (101h56m10.642321834s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.181707686 (101h56m10.643800159s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.182762434 (101h56m10.644854917s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.192828732 (101h56m10.654921205s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.198888852 (101h56m10.660981335s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.205614519 (101h56m10.667707012s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.207989521 (101h56m10.670081994s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.222032181 (101h56m10.684124654s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.290677625 (101h56m10.752770118s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.652827506 (101h56m11.114919989s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:52.180260159 (101h56m37.642352652s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180423099 (101h56m37.642515582s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180455370 (101h56m37.642547853s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180565359 (101h56m37.642657842s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:37784->77.90.31.15:4697: use of closed network connection; conn closed: op Sep 27 19:27:52.180610105 (101h56m37.642702588s) [seg_pos={0 0}, upload=89705]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 27 19:27:52.433487771 (101h56m37.895580254s) [upload=89705]: Done uploading segments... Sep 27 19:27:52.433517006 (101h56m37.89560948s) [upload=89705]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.269734332s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [................................................X....X..X.X.] duration: 6.257652477s FAIL (2025-09-27 19:28:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 27 19:27:22.886943307 (101h56m8.34903578s) [upload=89705]: Starting upload Sep 27 19:27:22.887049439 (101h56m8.349141922s) [upload=89705]: Done starting upload Sep 27 19:27:22.887160700 (101h56m8.349253184s) [upload=89705]: Uploading segments... Sep 27 19:27:22.887344050 (101h56m8.349436533s) [upload=89705]: Got next segment. Inline: false Sep 27 19:27:22.887393123 (101h56m8.349485606s) [upload=89705]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:27:22.887400948 (101h56m8.349493431s) [upload=89705]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:27:23.144152141 (101h56m8.606244624s) [seg_pos={0 0}, upload=89705]: Begin upload segment... Sep 27 19:27:23.144224288 (101h56m8.606316771s) [seg_pos={0 0}, upload=89705]: Done begin upload segment. Sep 27 19:27:23.144303519 (101h56m8.606396002s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144356600 (101h56m8.606449083s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144899694 (101h56m8.606992177s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145386381 (101h56m8.607478864s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145749413 (101h56m8.607841896s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146190593 (101h56m8.608283076s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146569114 (101h56m8.608661597s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146975998 (101h56m8.609068481s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.147437558 (101h56m8.609530051s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.148991526 (101h56m8.611083999s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.151370226 (101h56m8.613462699s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.153185462 (101h56m8.615277966s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.154735545 (101h56m8.616828028s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.158605123 (101h56m8.620697606s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162754133 (101h56m8.624846616s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162908967 (101h56m8.62500145s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:24.838034007 (101h56m10.30012649s) [upload=89705]: Next returned nil segment Sep 27 19:27:24.838098610 (101h56m10.300191093s) [upload=89705]: Waiting for error group managing segments... Sep 27 19:27:25.171482756 (101h56m10.633575229s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.176947972 (101h56m10.639040456s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.180229350 (101h56m10.642321834s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.181707686 (101h56m10.643800159s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.182762434 (101h56m10.644854917s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.192828732 (101h56m10.654921205s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.198888852 (101h56m10.660981335s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.205614519 (101h56m10.667707012s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.207989521 (101h56m10.670081994s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.222032181 (101h56m10.684124654s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.290677625 (101h56m10.752770118s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.652827506 (101h56m11.114919989s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:52.180260159 (101h56m37.642352652s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180423099 (101h56m37.642515582s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180455370 (101h56m37.642547853s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180565359 (101h56m37.642657842s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:37784->77.90.31.15:4697: use of closed network connection; conn closed: op Sep 27 19:27:52.180610105 (101h56m37.642702588s) [seg_pos={0 0}, upload=89705]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 27 19:27:52.433487771 (101h56m37.895580254s) [upload=89705]: Done uploading segments... Sep 27 19:27:52.433517006 (101h56m37.89560948s) [upload=89705]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.257683398s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................................................X....X..X.X] duration: 30.255567615s FAIL (2025-09-27 19:27:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 27 19:27:22.886943307 (101h56m8.34903578s) [upload=89705]: Starting upload Sep 27 19:27:22.887049439 (101h56m8.349141922s) [upload=89705]: Done starting upload Sep 27 19:27:22.887160700 (101h56m8.349253184s) [upload=89705]: Uploading segments... Sep 27 19:27:22.887344050 (101h56m8.349436533s) [upload=89705]: Got next segment. Inline: false Sep 27 19:27:22.887393123 (101h56m8.349485606s) [upload=89705]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:27:22.887400948 (101h56m8.349493431s) [upload=89705]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:27:23.144152141 (101h56m8.606244624s) [seg_pos={0 0}, upload=89705]: Begin upload segment... Sep 27 19:27:23.144224288 (101h56m8.606316771s) [seg_pos={0 0}, upload=89705]: Done begin upload segment. Sep 27 19:27:23.144303519 (101h56m8.606396002s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144356600 (101h56m8.606449083s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.144899694 (101h56m8.606992177s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145386381 (101h56m8.607478864s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.145749413 (101h56m8.607841896s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146190593 (101h56m8.608283076s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146569114 (101h56m8.608661597s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.146975998 (101h56m8.609068481s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.147437558 (101h56m8.609530051s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.148991526 (101h56m8.611083999s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.151370226 (101h56m8.613462699s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.153185462 (101h56m8.615277966s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.154735545 (101h56m8.616828028s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.158605123 (101h56m8.620697606s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162754133 (101h56m8.624846616s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:23.162908967 (101h56m8.62500145s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Uploading piece... Sep 27 19:27:24.838034007 (101h56m10.30012649s) [upload=89705]: Next returned nil segment Sep 27 19:27:24.838098610 (101h56m10.300191093s) [upload=89705]: Waiting for error group managing segments... Sep 27 19:27:25.171482756 (101h56m10.633575229s) [noise=true , address=145.79.227.19:4077 , piece_id=GIAYXZLLUURES3HHVNRQC6ZRQX2CZ5WDQASVHRPQPICA6PW4EQ2A, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.176947972 (101h56m10.639040456s) [noise=true , address=145.79.227.13:4803 , piece_id=TUT4CT2ORVPAKJKQP4J3QH5FZGFTB7PYRNXZ2XIQHUQ52EQDJXBA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.180229350 (101h56m10.642321834s) [noise=true , address=145.79.227.2:4115 , piece_id=JVWJRKDIFWN77BVDO4VPJIL6C4OO5BN7WBNN4TJO5SXKLFUTKV4Q, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.181707686 (101h56m10.643800159s) [noise=true , address=145.79.227.4:4187 , piece_id=WCGSKL7TB7VEZEPZ4ODOTTM5XEXN2YEJF37V6UFR4O2SB2XLCEWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.182762434 (101h56m10.644854917s) [noise=true , address=145.79.227.15:4875 , piece_id=XICWMTJP3OSIZXFHD3S2PUIWVTVULF36YFG2BMXPZYWK6PVJOALQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.192828732 (101h56m10.654921205s) [noise=true , address=145.79.227.3:4149 , piece_id=CQYYHMVQLWOLUDJIQJUILGP4TRPDF7NG6LXRCSZHG324RBGRCGZA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.198888852 (101h56m10.660981335s) [noise=true , address=145.79.227.14:4837 , piece_id=O65YIB773YZFS4CX3ENJYSDG2ILJHKAHMOQZARA6ILGCIMYFM7HA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.205614519 (101h56m10.667707012s) [noise=true , address=145.79.227.7:4295 , piece_id=YQB2LXXCSWTATSDT4C6JBCA2NWPDTAJ2BOL456U7HGV6WGSZRGDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.207989521 (101h56m10.670081994s) [noise=true , address=145.79.227.6:4259 , piece_id=QEEY7JI7PGB24S5CVWYOP6IMM5J7QYMRRIZVLVQK7ZDP2CGJYYYA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.222032181 (101h56m10.684124654s) [noise=true , address=145.79.227.11:4732 , piece_id=43B3XCLUV4GAZYCAXG56Y5I4S6L47LCVJL53HDLYODCS4OGOAOWQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.290677625 (101h56m10.752770118s) [noise=true , address=45.91.116.11:4118 , piece_id=WSNNWC5FALMJCWDOF6ORUJIXLK6G4PCKX3MTXMWSPBGRZ4SJKRRA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:25.652827506 (101h56m11.114919989s) [noise=true , address=45.91.116.9:4263 , piece_id=BV37BJAZJFVJ4RRXCFPEYKXTLYQ3JJDHSRCMYUXTSSHADFX2BZAQ, seg_pos={0 0}, upload=89705]: Done uploading piece. err: <nil> Sep 27 19:27:52.180260159 (101h56m37.642352652s) [noise=true , address=77.90.31.16:4734 , piece_id=CWEHVMSWIGUYFOSA5QQQIQYPUCLGNGMKYQEMF3J62DBXBMXXXXGA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180423099 (101h56m37.642515582s) [noise=true , address=77.90.31.14:4662 , piece_id=42PHYS3UEPXDUO3RATCUTQRQPXMS6HI3O6AOZI4R2NUULIKAWQVA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180455370 (101h56m37.642547853s) [noise=true , address=77.90.31.14:4661 , piece_id=KXFCG3HWMGGG5Z74VKNKEHRAM3VQSRUXK64OTLHGU5MNQS54IKDA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:27:52.180565359 (101h56m37.642657842s) [noise=true , address=77.90.31.15:4697 , piece_id=EWX3ABIK3WRK64RP4LHBJNXYTPI4N7YDRECFSG222PKPLQLZ3UWA, seg_pos={0 0}, upload=89705]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 188.166.225.170:37784->77.90.31.15:4697: use of closed network connection; conn closed: op Sep 27 19:27:52.180610105 (101h56m37.642702588s) [seg_pos={0 0}, upload=89705]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 27 19:27:52.433487771 (101h56m37.895580254s) [upload=89705]: Done uploading segments... Sep 27 19:27:52.433517006 (101h56m37.89560948s) [upload=89705]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.255593684s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................................................X....X..X.] duration: 6.404793089s FAIL (2025-09-27 19:26:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 27 19:26:06.666002390 (101h54m52.128094873s) [upload=89683]: Starting upload Sep 27 19:26:06.666063236 (101h54m52.128155709s) [upload=89683]: Done starting upload Sep 27 19:26:06.666233571 (101h54m52.128326064s) [upload=89683]: Uploading segments... Sep 27 19:26:06.666353238 (101h54m52.128445721s) [upload=89683]: Got next segment. Inline: false Sep 27 19:26:06.666392663 (101h54m52.128485146s) [upload=89683]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:26:06.666416529 (101h54m52.128509012s) [upload=89683]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:26:06.914833290 (101h54m52.376925763s) [seg_pos={0 0}, upload=89683]: Begin upload segment... Sep 27 19:26:06.914976173 (101h54m52.377068656s) [noise=true , address=45.91.116.7:4010 , piece_id=RFNQPBENUV6DVOOEF4SHE5UI5NIKDY7VTWQF7XXGKMP4PTKD7HHA, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.915026899 (101h54m52.377119392s) [seg_pos={0 0}, upload=89683]: Done begin upload segment. Sep 27 19:26:06.915071373 (101h54m52.377163856s) [noise=true , address=77.90.31.16:4734 , piece_id=TSHCA4NPVH7ASSXLH2IXFQH4MB3W4NUOVH6OJTVRXA5DFH7335BQ, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.916014951 (101h54m52.378107434s) [noise=true , address=145.79.227.15:4876 , piece_id=AKUHYKHEVAHYZJ3ZDK5AN5IMLJEN7CPNQ4ZDJMEBBBE63L3IW4BA, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.916841093 (101h54m52.378933576s) [noise=true , address=77.90.31.13:4628 , piece_id=OF472OLYROFV5P4VHOAYIQRNSSLVYGVKM546T4LUMKTKLWPNZXRQ, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.917948903 (101h54m52.380041386s) [noise=true , address=145.79.227.14:4840 , piece_id=3WYVZO3SNOI6P6YEOHRI7QS7F5I4Y2GO3CZMM3XAOZ4SHNEDQHFA, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.918010991 (101h54m52.380103465s) [noise=true , address=45.91.116.6:4975 , piece_id=T2LOKPCJKSXWSLALQ77YJB222W4BD2VSJKASV64TZWMXUJO5C5UQ, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.918681217 (101h54m52.38077369s) [noise=true , address=145.79.227.7:4294 , piece_id=PYT6UBFNW3TQ6KHZNJN7RABURXDYH6PIO7YTIRJH3Z2AYQDVL7GA, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.919382624 (101h54m52.381475107s) [noise=true , address=145.79.227.6:4258 , piece_id=DIMVULOXO2EIWU35RXPTHWTQ6KXT36P3I3KZ6FOPATGVKN3TQVEQ, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.920848625 (101h54m52.382941108s) [noise=true , address=145.79.227.11:4730 , piece_id=BZ3AWVICV4ICJFXSC7HPPIVK2MVKV5NNBY76LRN7SXD7SIQ6KQRQ, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.921949972 (101h54m52.384042445s) [noise=true , address=77.90.31.19:4843 , piece_id=3ZITATO3KE2D3MQ36K4A4FHJ7VTJN22ZIXYFHTTHL5KERHAERTJQ, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.923386979 (101h54m52.385479462s) [noise=true , address=77.90.31.21:4878 , piece_id=725LC3MVS4HRUNX4BZFRX3Z5XGPTP3VVZRS57S5L46ST433SVRIA, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.925334808 (101h54m52.387427281s) [noise=true , address=145.79.227.18:4005 , piece_id=FDVJEN6JZPBIXN6GH4VAJDG4CHZ5SS4ECN45YTXHTR6AXTCJ5FHA, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.926965111 (101h54m52.389057584s) [noise=true , address=145.79.227.12:4767 , piece_id=IAVWFIEZVBMNO6X2Y2AQMTGEC3VD3MPVFHL7GIIGZGLQYBH6LELA, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.929787866 (101h54m52.391880349s) [noise=true , address=145.79.227.13:4802 , piece_id=2Y6F4ZMVNE4DDLRKR56V3QLYP453Z76VURPC3N5V6OVRVU4CQ4KA, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.932958234 (101h54m52.395050717s) [noise=true , address=145.79.227.17:4972 , piece_id=ACSXFOOCIXRFJJNCGUDHMQ4KPHXPXXR6OVGFHY2OHTLGU4QH7T7A, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.936091581 (101h54m52.398184064s) [noise=true , address=145.79.227.17:4970 , piece_id=IZXGZHJWQKH6DFBKIE6BPURPTZAMILRV2WQPIR34AMPGL7EK36DA, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:08.662994064 (101h54m54.125086547s) [upload=89683]: Next returned nil segment Sep 27 19:26:08.663018440 (101h54m54.125110934s) [upload=89683]: Waiting for error group managing segments... Sep 27 19:26:08.949072931 (101h54m54.411165414s) [noise=true , address=145.79.227.6:4258 , piece_id=DIMVULOXO2EIWU35RXPTHWTQ6KXT36P3I3KZ6FOPATGVKN3TQVEQ, seg_pos={0 0}, upload=89683]: Done uploading piece. err: <nil> Sep 27 19:26:08.958939399 (101h54m54.421031872s) [noise=true , address=145.79.227.15:4876 , piece_id=AKUHYKHEVAHYZJ3ZDK5AN5IMLJEN7CPNQ4ZDJMEBBBE63L3IW4BA, seg_pos={0 0}, upload=89683]: Done uploading piece. err: <nil> Sep 27 19:26:08.963320953 (101h54m54.425413426s) [noise=true , address=145.79.227.17:4972 , piece_id=ACSXFOOCIXRFJJNCGUDHMQ4KPHXPXXR6OVGFHY2OHTLGU4QH7T7A, seg_pos={0 0}, upload=89683]: Done uploading piece. err: <nil> Sep 27 19:26:08.976137076 (101h54m54.43822956s) [noise=true , address=145.79.227.13:4802 , piece_id=2Y6F4ZMVNE4DDLRKR56V3QLYP453Z76VURPC3N5V6OVRVU4CQ4KA, seg_pos={0 0}, upload=89683]: Done uploading piece. err: <nil> Sep 27 19:26:08.976573919 (101h54m54.438666402s) [noise=true , address=145.79.227.14:4840 , piece_id=3WYVZO3SNOI6P6YEOHRI7QS7F5I4Y2GO3CZMM3XAOZ4SHNEDQHFA, seg_pos={0 0}, upload=89683]: Done uploading piece. err: <nil> Sep 27 19:26:08.980203109 (101h54m54.442295592s) [noise=true , address=145.79.227.11:4730 , piece_id=BZ3AWVICV4ICJFXSC7HPPIVK2MVKV5NNBY76LRN7SXD7SIQ6KQRQ, seg_pos={0 0}, upload=89683]: Done uploading piece. err: <nil> Sep 27 19:26:08.980699013 (101h54m54.442791496s) [noise=true , address=145.79.227.17:4970 , piece_id=IZXGZHJWQKH6DFBKIE6BPURPTZAMILRV2WQPIR34AMPGL7EK36DA, seg_pos={0 0}, upload=89683]: Done uploading piece. err: <nil> Sep 27 19:26:08.981674982 (101h54m54.443767465s) [noise=true , address=145.79.227.18:4005 , piece_id=FDVJEN6JZPBIXN6GH4VAJDG4CHZ5SS4ECN45YTXHTR6AXTCJ5FHA, seg_pos={0 0}, upload=89683]: Done uploading piece. err: <nil> Sep 27 19:26:09.345618488 (101h54m54.807710961s) [noise=true , address=45.91.116.7:4010 , piece_id=RFNQPBENUV6DVOOEF4SHE5UI5NIKDY7VTWQF7XXGKMP4PTKD7HHA, seg_pos={0 0}, upload=89683]: Done uploading piece. err: <nil> Sep 27 19:26:09.351055431 (101h54m54.813147914s) [noise=true , address=145.79.227.12:4767 , piece_id=IAVWFIEZVBMNO6X2Y2AQMTGEC3VD3MPVFHL7GIIGZGLQYBH6LELA, seg_pos={0 0}, upload=89683]: Done uploading piece. err: <nil> Sep 27 19:26:09.531216231 (101h54m54.993308704s) [noise=true , address=45.91.116.6:4975 , piece_id=T2LOKPCJKSXWSLALQ77YJB222W4BD2VSJKASV64TZWMXUJO5C5UQ, seg_pos={0 0}, upload=89683]: Done uploading piece. err: <nil> Sep 27 19:26:12.173971690 (101h54m57.636064173s) [noise=true , address=145.79.227.7:4294 , piece_id=PYT6UBFNW3TQ6KHZNJN7RABURXDYH6PIO7YTIRJH3Z2AYQDVL7GA, seg_pos={0 0}, upload=89683]: Done uploading piece. err: <nil> Sep 27 19:26:35.962560617 (101h55m21.4246531s) [noise=true , address=77.90.31.16:4734 , piece_id=TSHCA4NPVH7ASSXLH2IXFQH4MB3W4NUOVH6OJTVRXA5DFH7335BQ, seg_pos={0 0}, upload=89683]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:26:35.962704862 (101h55m21.424797345s) [noise=true , address=77.90.31.19:4843 , piece_id=3ZITATO3KE2D3MQ36K4A4FHJ7VTJN22ZIXYFHTTHL5KERHAERTJQ, seg_pos={0 0}, upload=89683]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:26:35.962787990 (101h55m21.424880463s) [noise=true , address=77.90.31.21:4878 , piece_id=725LC3MVS4HRUNX4BZFRX3Z5XGPTP3VVZRS57S5L46ST433SVRIA, seg_pos={0 0}, upload=89683]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: write tcp 188.166.225.170:57458->77.90.31.21:4878: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:26:35.962912678 (101h55m21.425005151s) [noise=true , address=77.90.31.13:4628 , piece_id=OF472OLYROFV5P4VHOAYIQRNSSLVYGVKM546T4LUMKTKLWPNZXRQ, seg_pos={0 0}, upload=89683]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: context deadline exceeded; context deadline exceeded Sep 27 19:26:35.962944258 (101h55m21.425036741s) [seg_pos={0 0}, upload=89683]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 27 19:26:36.217280131 (101h55m21.679372604s) [upload=89683]: Done uploading segments... Sep 27 19:26:36.217308906 (101h55m21.679401389s) [upload=89683]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.404811413s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................................................X....X..X] duration: 30.256402891s FAIL (2025-09-27 19:26:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 27 19:26:06.666002390 (101h54m52.128094873s) [upload=89683]: Starting upload Sep 27 19:26:06.666063236 (101h54m52.128155709s) [upload=89683]: Done starting upload Sep 27 19:26:06.666233571 (101h54m52.128326064s) [upload=89683]: Uploading segments... Sep 27 19:26:06.666353238 (101h54m52.128445721s) [upload=89683]: Got next segment. Inline: false Sep 27 19:26:06.666392663 (101h54m52.128485146s) [upload=89683]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:26:06.666416529 (101h54m52.128509012s) [upload=89683]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:26:06.914833290 (101h54m52.376925763s) [seg_pos={0 0}, upload=89683]: Begin upload segment... Sep 27 19:26:06.914976173 (101h54m52.377068656s) [noise=true , address=45.91.116.7:4010 , piece_id=RFNQPBENUV6DVOOEF4SHE5UI5NIKDY7VTWQF7XXGKMP4PTKD7HHA, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.915026899 (101h54m52.377119392s) [seg_pos={0 0}, upload=89683]: Done begin upload segment. Sep 27 19:26:06.915071373 (101h54m52.377163856s) [noise=true , address=77.90.31.16:4734 , piece_id=TSHCA4NPVH7ASSXLH2IXFQH4MB3W4NUOVH6OJTVRXA5DFH7335BQ, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.916014951 (101h54m52.378107434s) [noise=true , address=145.79.227.15:4876 , piece_id=AKUHYKHEVAHYZJ3ZDK5AN5IMLJEN7CPNQ4ZDJMEBBBE63L3IW4BA, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.916841093 (101h54m52.378933576s) [noise=true , address=77.90.31.13:4628 , piece_id=OF472OLYROFV5P4VHOAYIQRNSSLVYGVKM546T4LUMKTKLWPNZXRQ, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.917948903 (101h54m52.380041386s) [noise=true , address=145.79.227.14:4840 , piece_id=3WYVZO3SNOI6P6YEOHRI7QS7F5I4Y2GO3CZMM3XAOZ4SHNEDQHFA, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.918010991 (101h54m52.380103465s) [noise=true , address=45.91.116.6:4975 , piece_id=T2LOKPCJKSXWSLALQ77YJB222W4BD2VSJKASV64TZWMXUJO5C5UQ, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.918681217 (101h54m52.38077369s) [noise=true , address=145.79.227.7:4294 , piece_id=PYT6UBFNW3TQ6KHZNJN7RABURXDYH6PIO7YTIRJH3Z2AYQDVL7GA, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.919382624 (101h54m52.381475107s) [noise=true , address=145.79.227.6:4258 , piece_id=DIMVULOXO2EIWU35RXPTHWTQ6KXT36P3I3KZ6FOPATGVKN3TQVEQ, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.920848625 (101h54m52.382941108s) [noise=true , address=145.79.227.11:4730 , piece_id=BZ3AWVICV4ICJFXSC7HPPIVK2MVKV5NNBY76LRN7SXD7SIQ6KQRQ, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.921949972 (101h54m52.384042445s) [noise=true , address=77.90.31.19:4843 , piece_id=3ZITATO3KE2D3MQ36K4A4FHJ7VTJN22ZIXYFHTTHL5KERHAERTJQ, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.923386979 (101h54m52.385479462s) [noise=true , address=77.90.31.21:4878 , piece_id=725LC3MVS4HRUNX4BZFRX3Z5XGPTP3VVZRS57S5L46ST433SVRIA, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.925334808 (101h54m52.387427281s) [noise=true , address=145.79.227.18:4005 , piece_id=FDVJEN6JZPBIXN6GH4VAJDG4CHZ5SS4ECN45YTXHTR6AXTCJ5FHA, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.926965111 (101h54m52.389057584s) [noise=true , address=145.79.227.12:4767 , piece_id=IAVWFIEZVBMNO6X2Y2AQMTGEC3VD3MPVFHL7GIIGZGLQYBH6LELA, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.929787866 (101h54m52.391880349s) [noise=true , address=145.79.227.13:4802 , piece_id=2Y6F4ZMVNE4DDLRKR56V3QLYP453Z76VURPC3N5V6OVRVU4CQ4KA, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.932958234 (101h54m52.395050717s) [noise=true , address=145.79.227.17:4972 , piece_id=ACSXFOOCIXRFJJNCGUDHMQ4KPHXPXXR6OVGFHY2OHTLGU4QH7T7A, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:06.936091581 (101h54m52.398184064s) [noise=true , address=145.79.227.17:4970 , piece_id=IZXGZHJWQKH6DFBKIE6BPURPTZAMILRV2WQPIR34AMPGL7EK36DA, seg_pos={0 0}, upload=89683]: Uploading piece... Sep 27 19:26:08.662994064 (101h54m54.125086547s) [upload=89683]: Next returned nil segment Sep 27 19:26:08.663018440 (101h54m54.125110934s) [upload=89683]: Waiting for error group managing segments... Sep 27 19:26:08.949072931 (101h54m54.411165414s) [noise=true , address=145.79.227.6:4258 , piece_id=DIMVULOXO2EIWU35RXPTHWTQ6KXT36P3I3KZ6FOPATGVKN3TQVEQ, seg_pos={0 0}, upload=89683]: Done uploading piece. err: <nil> Sep 27 19:26:08.958939399 (101h54m54.421031872s) [noise=true , address=145.79.227.15:4876 , piece_id=AKUHYKHEVAHYZJ3ZDK5AN5IMLJEN7CPNQ4ZDJMEBBBE63L3IW4BA, seg_pos={0 0}, upload=89683]: Done uploading piece. err: <nil> Sep 27 19:26:08.963320953 (101h54m54.425413426s) [noise=true , address=145.79.227.17:4972 , piece_id=ACSXFOOCIXRFJJNCGUDHMQ4KPHXPXXR6OVGFHY2OHTLGU4QH7T7A, seg_pos={0 0}, upload=89683]: Done uploading piece. err: <nil> Sep 27 19:26:08.976137076 (101h54m54.43822956s) [noise=true , address=145.79.227.13:4802 , piece_id=2Y6F4ZMVNE4DDLRKR56V3QLYP453Z76VURPC3N5V6OVRVU4CQ4KA, seg_pos={0 0}, upload=89683]: Done uploading piece. err: <nil> Sep 27 19:26:08.976573919 (101h54m54.438666402s) [noise=true , address=145.79.227.14:4840 , piece_id=3WYVZO3SNOI6P6YEOHRI7QS7F5I4Y2GO3CZMM3XAOZ4SHNEDQHFA, seg_pos={0 0}, upload=89683]: Done uploading piece. err: <nil> Sep 27 19:26:08.980203109 (101h54m54.442295592s) [noise=true , address=145.79.227.11:4730 , piece_id=BZ3AWVICV4ICJFXSC7HPPIVK2MVKV5NNBY76LRN7SXD7SIQ6KQRQ, seg_pos={0 0}, upload=89683]: Done uploading piece. err: <nil> Sep 27 19:26:08.980699013 (101h54m54.442791496s) [noise=true , address=145.79.227.17:4970 , piece_id=IZXGZHJWQKH6DFBKIE6BPURPTZAMILRV2WQPIR34AMPGL7EK36DA, seg_pos={0 0}, upload=89683]: Done uploading piece. err: <nil> Sep 27 19:26:08.981674982 (101h54m54.443767465s) [noise=true , address=145.79.227.18:4005 , piece_id=FDVJEN6JZPBIXN6GH4VAJDG4CHZ5SS4ECN45YTXHTR6AXTCJ5FHA, seg_pos={0 0}, upload=89683]: Done uploading piece. err: <nil> Sep 27 19:26:09.345618488 (101h54m54.807710961s) [noise=true , address=45.91.116.7:4010 , piece_id=RFNQPBENUV6DVOOEF4SHE5UI5NIKDY7VTWQF7XXGKMP4PTKD7HHA, seg_pos={0 0}, upload=89683]: Done uploading piece. err: <nil> Sep 27 19:26:09.351055431 (101h54m54.813147914s) [noise=true , address=145.79.227.12:4767 , piece_id=IAVWFIEZVBMNO6X2Y2AQMTGEC3VD3MPVFHL7GIIGZGLQYBH6LELA, seg_pos={0 0}, upload=89683]: Done uploading piece. err: <nil> Sep 27 19:26:09.531216231 (101h54m54.993308704s) [noise=true , address=45.91.116.6:4975 , piece_id=T2LOKPCJKSXWSLALQ77YJB222W4BD2VSJKASV64TZWMXUJO5C5UQ, seg_pos={0 0}, upload=89683]: Done uploading piece. err: <nil> Sep 27 19:26:12.173971690 (101h54m57.636064173s) [noise=true , address=145.79.227.7:4294 , piece_id=PYT6UBFNW3TQ6KHZNJN7RABURXDYH6PIO7YTIRJH3Z2AYQDVL7GA, seg_pos={0 0}, upload=89683]: Done uploading piece. err: <nil> Sep 27 19:26:35.962560617 (101h55m21.4246531s) [noise=true , address=77.90.31.16:4734 , piece_id=TSHCA4NPVH7ASSXLH2IXFQH4MB3W4NUOVH6OJTVRXA5DFH7335BQ, seg_pos={0 0}, upload=89683]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:26:35.962704862 (101h55m21.424797345s) [noise=true , address=77.90.31.19:4843 , piece_id=3ZITATO3KE2D3MQ36K4A4FHJ7VTJN22ZIXYFHTTHL5KERHAERTJQ, seg_pos={0 0}, upload=89683]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:26:35.962787990 (101h55m21.424880463s) [noise=true , address=77.90.31.21:4878 , piece_id=725LC3MVS4HRUNX4BZFRX3Z5XGPTP3VVZRS57S5L46ST433SVRIA, seg_pos={0 0}, upload=89683]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: write tcp 188.166.225.170:57458->77.90.31.21:4878: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:26:35.962912678 (101h55m21.425005151s) [noise=true , address=77.90.31.13:4628 , piece_id=OF472OLYROFV5P4VHOAYIQRNSSLVYGVKM546T4LUMKTKLWPNZXRQ, seg_pos={0 0}, upload=89683]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: context deadline exceeded; context deadline exceeded Sep 27 19:26:35.962944258 (101h55m21.425036741s) [seg_pos={0 0}, upload=89683]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 27 19:26:36.217280131 (101h55m21.679372604s) [upload=89683]: Done uploading segments... Sep 27 19:26:36.217308906 (101h55m21.679401389s) [upload=89683]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.256417338s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................................X....X..] duration: 8.979604882s FAIL (2025-09-27 19:25:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 27 19:23:00.713190232 (101h51m46.175282715s) [upload=89637]: Starting upload Sep 27 19:23:00.713302665 (101h51m46.175395149s) [upload=89637]: Done starting upload Sep 27 19:23:00.713465546 (101h51m46.175558029s) [upload=89637]: Uploading segments... Sep 27 19:23:00.713623217 (101h51m46.1757157s) [upload=89637]: Got next segment. Inline: false Sep 27 19:23:00.713687077 (101h51m46.17577955s) [upload=89637]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:23:00.713694482 (101h51m46.175786965s) [upload=89637]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:23:01.000652494 (101h51m46.462745007s) [seg_pos={0 0}, upload=89637]: Begin upload segment... Sep 27 19:23:01.000777873 (101h51m46.462870356s) [seg_pos={0 0}, upload=89637]: Done begin upload segment. Sep 27 19:23:01.000844339 (101h51m46.462936852s) [noise=true , address=45.91.116.12:4225 , piece_id=XI4P63YFEIFS3EZROLQDQUM2GD522FFKABLJ4PGOFY2QKWRF4WZQ, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.000872884 (101h51m46.462965377s) [noise=true , address=77.90.31.15:4697 , piece_id=RYZ5IY6FWLNNRLF2CLJ7RYJOSONBTEIBVBVNFM5ASOTSJ57LFDNQ, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.001674079 (101h51m46.463766583s) [noise=true , address=145.79.227.17:4972 , piece_id=WDKM3UXNVR374D3CUSPCN23G332LYP6HBZADBW2D73WOTKRCXU7Q, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.002228485 (101h51m46.464320988s) [noise=true , address=145.79.227.16:4911 , piece_id=PWUJSPH2LNCIEMYULINGYKNJS6K4VL57FL26Z6ROT6RAXOJBWKDA, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.002616734 (101h51m46.464709217s) [noise=true , address=45.91.116.12:4226 , piece_id=FRXDV3XG7K5UVQ3AFS2CKHNJK36VCF3JD6G6PK7SLEU4P7VCFOTA, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.003063555 (101h51m46.465156038s) [noise=true , address=145.79.227.12:4765 , piece_id=VRNQHMIEDOTON4SPDVBBJ3HQTPCSAOGV435XWJ35RPD6RF275RTQ, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.003789919 (101h51m46.465882422s) [noise=true , address=77.90.31.18:4807 , piece_id=GFNGQAB7Z3F4DEZCNIVIIED5S3QDTQU5G5CTWKTTDKO2TT3XTFSQ, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.005679176 (101h51m46.467771659s) [noise=true , address=45.91.116.9:4262 , piece_id=TBA6RMCZS67TSX555MURJJMTLLKLG43VJ6RXNTAVDR6YD73GEJ3Q, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.005709454 (101h51m46.467801947s) [noise=true , address=145.79.227.18:4007 , piece_id=VA4U55INCQJXY733KXRADUJHTDF3FIXGYKLFF6VJCIJVFV5FBKFQ, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.007226572 (101h51m46.469319065s) [noise=true , address=45.91.116.10:4082 , piece_id=M7D7MIJSPQCBXAHADPH6SPDODVJSLRDCTW5NI5SCPHHATV2BNA4A, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.011411100 (101h51m46.473503583s) [noise=true , address=145.79.227.13:4804 , piece_id=DMXTP2YNQUF5JMO4GWQHYGRYRXBSYS3EIXHX5OJIXCRA6FCMT6WA, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.014674504 (101h51m46.476766987s) [noise=true , address=77.90.31.19:4843 , piece_id=LJXNSV6TCVQF4FEASDLBJG7UHGQ5J3F2AOILPQ2PTPMERQT34ECA, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.020418301 (101h51m46.482510824s) [noise=true , address=145.79.227.7:4293 , piece_id=42VM53KR7S7HWHIVQPUUNNRFX5OVT4GBN3BKKQOFH7UJIBJHQN4A, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.022297389 (101h51m46.484389872s) [noise=true , address=45.91.116.5:4299 , piece_id=FZMU5DZZYREA4MXMLZIH6PXUDAKSSH7MZVD35SOQ7TPARF6MT5PQ, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.030930067 (101h51m46.493022561s) [noise=true , address=145.79.227.3:4149 , piece_id=CGXWXBPPQVOCVFVSQ576G3CQXNKHK7FI2776IEKKACUSRQREOMOA, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.031244316 (101h51m46.493336809s) [noise=true , address=145.79.227.16:4912 , piece_id=BFHV5JIFMCTMMMXVIWIWQIDNLAGWC2LAWHDCYCAFF3WKHRTFIJPQ, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:02.765616825 (101h51m48.227709298s) [upload=89637]: Next returned nil segment Sep 27 19:23:02.765707788 (101h51m48.227800271s) [upload=89637]: Waiting for error group managing segments... Sep 27 19:23:03.050942347 (101h51m48.51303483s) [noise=true , address=145.79.227.17:4972 , piece_id=WDKM3UXNVR374D3CUSPCN23G332LYP6HBZADBW2D73WOTKRCXU7Q, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:03.066525011 (101h51m48.528617494s) [noise=true , address=145.79.227.16:4912 , piece_id=BFHV5JIFMCTMMMXVIWIWQIDNLAGWC2LAWHDCYCAFF3WKHRTFIJPQ, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:03.069000955 (101h51m48.531093428s) [noise=true , address=145.79.227.12:4765 , piece_id=VRNQHMIEDOTON4SPDVBBJ3HQTPCSAOGV435XWJ35RPD6RF275RTQ, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:03.074882284 (101h51m48.536974767s) [noise=true , address=145.79.227.16:4911 , piece_id=PWUJSPH2LNCIEMYULINGYKNJS6K4VL57FL26Z6ROT6RAXOJBWKDA, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:03.080427743 (101h51m48.542520226s) [noise=true , address=145.79.227.13:4804 , piece_id=DMXTP2YNQUF5JMO4GWQHYGRYRXBSYS3EIXHX5OJIXCRA6FCMT6WA, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:03.085761279 (101h51m48.547853762s) [noise=true , address=145.79.227.3:4149 , piece_id=CGXWXBPPQVOCVFVSQ576G3CQXNKHK7FI2776IEKKACUSRQREOMOA, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:03.106344614 (101h51m48.568437097s) [noise=true , address=145.79.227.18:4007 , piece_id=VA4U55INCQJXY733KXRADUJHTDF3FIXGYKLFF6VJCIJVFV5FBKFQ, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:03.186240711 (101h51m48.648333194s) [noise=true , address=77.90.31.15:4697 , piece_id=RYZ5IY6FWLNNRLF2CLJ7RYJOSONBTEIBVBVNFM5ASOTSJ57LFDNQ, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:03.284507278 (101h51m48.746599752s) [noise=true , address=77.90.31.18:4807 , piece_id=GFNGQAB7Z3F4DEZCNIVIIED5S3QDTQU5G5CTWKTTDKO2TT3XTFSQ, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:03.304107621 (101h51m48.766200104s) [noise=true , address=77.90.31.19:4843 , piece_id=LJXNSV6TCVQF4FEASDLBJG7UHGQ5J3F2AOILPQ2PTPMERQT34ECA, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:04.442365286 (101h51m49.90445777s) [noise=true , address=145.79.227.7:4293 , piece_id=42VM53KR7S7HWHIVQPUUNNRFX5OVT4GBN3BKKQOFH7UJIBJHQN4A, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:29.984729682 (101h52m15.446822155s) [noise=true , address=45.91.116.10:4082 , piece_id=M7D7MIJSPQCBXAHADPH6SPDODVJSLRDCTW5NI5SCPHHATV2BNA4A, seg_pos={0 0}, upload=89637]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: rpc: dial tcp 45.91.116.10:4082: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:23:29.984885788 (101h52m15.446978261s) [noise=true , address=45.91.116.12:4225 , piece_id=XI4P63YFEIFS3EZROLQDQUM2GD522FFKABLJ4PGOFY2QKWRF4WZQ, seg_pos={0 0}, upload=89637]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:23:29.985001008 (101h52m15.447093481s) [noise=true , address=45.91.116.12:4226 , piece_id=FRXDV3XG7K5UVQ3AFS2CKHNJK36VCF3JD6G6PK7SLEU4P7VCFOTA, seg_pos={0 0}, upload=89637]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:23:29.985137719 (101h52m15.447230212s) [noise=true , address=45.91.116.5:4299 , piece_id=FZMU5DZZYREA4MXMLZIH6PXUDAKSSH7MZVD35SOQ7TPARF6MT5PQ, seg_pos={0 0}, upload=89637]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:23:29.985432991 (101h52m15.447525474s) [noise=true , address=45.91.116.9:4262 , piece_id=TBA6RMCZS67TSX555MURJJMTLLKLG43VJ6RXNTAVDR6YD73GEJ3Q, seg_pos={0 0}, upload=89637]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 188.166.225.170:35588->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:23:29.985464801 (101h52m15.447557284s) [seg_pos={0 0}, upload=89637]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 27 19:23:30.235762735 (101h52m15.697855209s) [upload=89637]: Done uploading segments... Sep 27 19:23:30.235800337 (101h52m15.69789282s) [upload=89637]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 8.979646582s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................X....X.] duration: 6.090962508s FAIL (2025-09-27 19:24:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 27 19:23:00.713190232 (101h51m46.175282715s) [upload=89637]: Starting upload Sep 27 19:23:00.713302665 (101h51m46.175395149s) [upload=89637]: Done starting upload Sep 27 19:23:00.713465546 (101h51m46.175558029s) [upload=89637]: Uploading segments... Sep 27 19:23:00.713623217 (101h51m46.1757157s) [upload=89637]: Got next segment. Inline: false Sep 27 19:23:00.713687077 (101h51m46.17577955s) [upload=89637]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:23:00.713694482 (101h51m46.175786965s) [upload=89637]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:23:01.000652494 (101h51m46.462745007s) [seg_pos={0 0}, upload=89637]: Begin upload segment... Sep 27 19:23:01.000777873 (101h51m46.462870356s) [seg_pos={0 0}, upload=89637]: Done begin upload segment. Sep 27 19:23:01.000844339 (101h51m46.462936852s) [noise=true , address=45.91.116.12:4225 , piece_id=XI4P63YFEIFS3EZROLQDQUM2GD522FFKABLJ4PGOFY2QKWRF4WZQ, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.000872884 (101h51m46.462965377s) [noise=true , address=77.90.31.15:4697 , piece_id=RYZ5IY6FWLNNRLF2CLJ7RYJOSONBTEIBVBVNFM5ASOTSJ57LFDNQ, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.001674079 (101h51m46.463766583s) [noise=true , address=145.79.227.17:4972 , piece_id=WDKM3UXNVR374D3CUSPCN23G332LYP6HBZADBW2D73WOTKRCXU7Q, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.002228485 (101h51m46.464320988s) [noise=true , address=145.79.227.16:4911 , piece_id=PWUJSPH2LNCIEMYULINGYKNJS6K4VL57FL26Z6ROT6RAXOJBWKDA, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.002616734 (101h51m46.464709217s) [noise=true , address=45.91.116.12:4226 , piece_id=FRXDV3XG7K5UVQ3AFS2CKHNJK36VCF3JD6G6PK7SLEU4P7VCFOTA, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.003063555 (101h51m46.465156038s) [noise=true , address=145.79.227.12:4765 , piece_id=VRNQHMIEDOTON4SPDVBBJ3HQTPCSAOGV435XWJ35RPD6RF275RTQ, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.003789919 (101h51m46.465882422s) [noise=true , address=77.90.31.18:4807 , piece_id=GFNGQAB7Z3F4DEZCNIVIIED5S3QDTQU5G5CTWKTTDKO2TT3XTFSQ, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.005679176 (101h51m46.467771659s) [noise=true , address=45.91.116.9:4262 , piece_id=TBA6RMCZS67TSX555MURJJMTLLKLG43VJ6RXNTAVDR6YD73GEJ3Q, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.005709454 (101h51m46.467801947s) [noise=true , address=145.79.227.18:4007 , piece_id=VA4U55INCQJXY733KXRADUJHTDF3FIXGYKLFF6VJCIJVFV5FBKFQ, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.007226572 (101h51m46.469319065s) [noise=true , address=45.91.116.10:4082 , piece_id=M7D7MIJSPQCBXAHADPH6SPDODVJSLRDCTW5NI5SCPHHATV2BNA4A, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.011411100 (101h51m46.473503583s) [noise=true , address=145.79.227.13:4804 , piece_id=DMXTP2YNQUF5JMO4GWQHYGRYRXBSYS3EIXHX5OJIXCRA6FCMT6WA, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.014674504 (101h51m46.476766987s) [noise=true , address=77.90.31.19:4843 , piece_id=LJXNSV6TCVQF4FEASDLBJG7UHGQ5J3F2AOILPQ2PTPMERQT34ECA, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.020418301 (101h51m46.482510824s) [noise=true , address=145.79.227.7:4293 , piece_id=42VM53KR7S7HWHIVQPUUNNRFX5OVT4GBN3BKKQOFH7UJIBJHQN4A, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.022297389 (101h51m46.484389872s) [noise=true , address=45.91.116.5:4299 , piece_id=FZMU5DZZYREA4MXMLZIH6PXUDAKSSH7MZVD35SOQ7TPARF6MT5PQ, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.030930067 (101h51m46.493022561s) [noise=true , address=145.79.227.3:4149 , piece_id=CGXWXBPPQVOCVFVSQ576G3CQXNKHK7FI2776IEKKACUSRQREOMOA, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.031244316 (101h51m46.493336809s) [noise=true , address=145.79.227.16:4912 , piece_id=BFHV5JIFMCTMMMXVIWIWQIDNLAGWC2LAWHDCYCAFF3WKHRTFIJPQ, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:02.765616825 (101h51m48.227709298s) [upload=89637]: Next returned nil segment Sep 27 19:23:02.765707788 (101h51m48.227800271s) [upload=89637]: Waiting for error group managing segments... Sep 27 19:23:03.050942347 (101h51m48.51303483s) [noise=true , address=145.79.227.17:4972 , piece_id=WDKM3UXNVR374D3CUSPCN23G332LYP6HBZADBW2D73WOTKRCXU7Q, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:03.066525011 (101h51m48.528617494s) [noise=true , address=145.79.227.16:4912 , piece_id=BFHV5JIFMCTMMMXVIWIWQIDNLAGWC2LAWHDCYCAFF3WKHRTFIJPQ, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:03.069000955 (101h51m48.531093428s) [noise=true , address=145.79.227.12:4765 , piece_id=VRNQHMIEDOTON4SPDVBBJ3HQTPCSAOGV435XWJ35RPD6RF275RTQ, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:03.074882284 (101h51m48.536974767s) [noise=true , address=145.79.227.16:4911 , piece_id=PWUJSPH2LNCIEMYULINGYKNJS6K4VL57FL26Z6ROT6RAXOJBWKDA, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:03.080427743 (101h51m48.542520226s) [noise=true , address=145.79.227.13:4804 , piece_id=DMXTP2YNQUF5JMO4GWQHYGRYRXBSYS3EIXHX5OJIXCRA6FCMT6WA, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:03.085761279 (101h51m48.547853762s) [noise=true , address=145.79.227.3:4149 , piece_id=CGXWXBPPQVOCVFVSQ576G3CQXNKHK7FI2776IEKKACUSRQREOMOA, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:03.106344614 (101h51m48.568437097s) [noise=true , address=145.79.227.18:4007 , piece_id=VA4U55INCQJXY733KXRADUJHTDF3FIXGYKLFF6VJCIJVFV5FBKFQ, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:03.186240711 (101h51m48.648333194s) [noise=true , address=77.90.31.15:4697 , piece_id=RYZ5IY6FWLNNRLF2CLJ7RYJOSONBTEIBVBVNFM5ASOTSJ57LFDNQ, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:03.284507278 (101h51m48.746599752s) [noise=true , address=77.90.31.18:4807 , piece_id=GFNGQAB7Z3F4DEZCNIVIIED5S3QDTQU5G5CTWKTTDKO2TT3XTFSQ, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:03.304107621 (101h51m48.766200104s) [noise=true , address=77.90.31.19:4843 , piece_id=LJXNSV6TCVQF4FEASDLBJG7UHGQ5J3F2AOILPQ2PTPMERQT34ECA, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:04.442365286 (101h51m49.90445777s) [noise=true , address=145.79.227.7:4293 , piece_id=42VM53KR7S7HWHIVQPUUNNRFX5OVT4GBN3BKKQOFH7UJIBJHQN4A, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:29.984729682 (101h52m15.446822155s) [noise=true , address=45.91.116.10:4082 , piece_id=M7D7MIJSPQCBXAHADPH6SPDODVJSLRDCTW5NI5SCPHHATV2BNA4A, seg_pos={0 0}, upload=89637]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: rpc: dial tcp 45.91.116.10:4082: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:23:29.984885788 (101h52m15.446978261s) [noise=true , address=45.91.116.12:4225 , piece_id=XI4P63YFEIFS3EZROLQDQUM2GD522FFKABLJ4PGOFY2QKWRF4WZQ, seg_pos={0 0}, upload=89637]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:23:29.985001008 (101h52m15.447093481s) [noise=true , address=45.91.116.12:4226 , piece_id=FRXDV3XG7K5UVQ3AFS2CKHNJK36VCF3JD6G6PK7SLEU4P7VCFOTA, seg_pos={0 0}, upload=89637]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:23:29.985137719 (101h52m15.447230212s) [noise=true , address=45.91.116.5:4299 , piece_id=FZMU5DZZYREA4MXMLZIH6PXUDAKSSH7MZVD35SOQ7TPARF6MT5PQ, seg_pos={0 0}, upload=89637]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:23:29.985432991 (101h52m15.447525474s) [noise=true , address=45.91.116.9:4262 , piece_id=TBA6RMCZS67TSX555MURJJMTLLKLG43VJ6RXNTAVDR6YD73GEJ3Q, seg_pos={0 0}, upload=89637]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 188.166.225.170:35588->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:23:29.985464801 (101h52m15.447557284s) [seg_pos={0 0}, upload=89637]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 27 19:23:30.235762735 (101h52m15.697855209s) [upload=89637]: Done uploading segments... Sep 27 19:23:30.235800337 (101h52m15.69789282s) [upload=89637]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.091008316s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................................X....X] duration: 30.252722505s FAIL (2025-09-27 19:23:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 27 19:23:00.713190232 (101h51m46.175282715s) [upload=89637]: Starting upload Sep 27 19:23:00.713302665 (101h51m46.175395149s) [upload=89637]: Done starting upload Sep 27 19:23:00.713465546 (101h51m46.175558029s) [upload=89637]: Uploading segments... Sep 27 19:23:00.713623217 (101h51m46.1757157s) [upload=89637]: Got next segment. Inline: false Sep 27 19:23:00.713687077 (101h51m46.17577955s) [upload=89637]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:23:00.713694482 (101h51m46.175786965s) [upload=89637]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:23:01.000652494 (101h51m46.462745007s) [seg_pos={0 0}, upload=89637]: Begin upload segment... Sep 27 19:23:01.000777873 (101h51m46.462870356s) [seg_pos={0 0}, upload=89637]: Done begin upload segment. Sep 27 19:23:01.000844339 (101h51m46.462936852s) [noise=true , address=45.91.116.12:4225 , piece_id=XI4P63YFEIFS3EZROLQDQUM2GD522FFKABLJ4PGOFY2QKWRF4WZQ, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.000872884 (101h51m46.462965377s) [noise=true , address=77.90.31.15:4697 , piece_id=RYZ5IY6FWLNNRLF2CLJ7RYJOSONBTEIBVBVNFM5ASOTSJ57LFDNQ, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.001674079 (101h51m46.463766583s) [noise=true , address=145.79.227.17:4972 , piece_id=WDKM3UXNVR374D3CUSPCN23G332LYP6HBZADBW2D73WOTKRCXU7Q, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.002228485 (101h51m46.464320988s) [noise=true , address=145.79.227.16:4911 , piece_id=PWUJSPH2LNCIEMYULINGYKNJS6K4VL57FL26Z6ROT6RAXOJBWKDA, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.002616734 (101h51m46.464709217s) [noise=true , address=45.91.116.12:4226 , piece_id=FRXDV3XG7K5UVQ3AFS2CKHNJK36VCF3JD6G6PK7SLEU4P7VCFOTA, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.003063555 (101h51m46.465156038s) [noise=true , address=145.79.227.12:4765 , piece_id=VRNQHMIEDOTON4SPDVBBJ3HQTPCSAOGV435XWJ35RPD6RF275RTQ, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.003789919 (101h51m46.465882422s) [noise=true , address=77.90.31.18:4807 , piece_id=GFNGQAB7Z3F4DEZCNIVIIED5S3QDTQU5G5CTWKTTDKO2TT3XTFSQ, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.005679176 (101h51m46.467771659s) [noise=true , address=45.91.116.9:4262 , piece_id=TBA6RMCZS67TSX555MURJJMTLLKLG43VJ6RXNTAVDR6YD73GEJ3Q, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.005709454 (101h51m46.467801947s) [noise=true , address=145.79.227.18:4007 , piece_id=VA4U55INCQJXY733KXRADUJHTDF3FIXGYKLFF6VJCIJVFV5FBKFQ, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.007226572 (101h51m46.469319065s) [noise=true , address=45.91.116.10:4082 , piece_id=M7D7MIJSPQCBXAHADPH6SPDODVJSLRDCTW5NI5SCPHHATV2BNA4A, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.011411100 (101h51m46.473503583s) [noise=true , address=145.79.227.13:4804 , piece_id=DMXTP2YNQUF5JMO4GWQHYGRYRXBSYS3EIXHX5OJIXCRA6FCMT6WA, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.014674504 (101h51m46.476766987s) [noise=true , address=77.90.31.19:4843 , piece_id=LJXNSV6TCVQF4FEASDLBJG7UHGQ5J3F2AOILPQ2PTPMERQT34ECA, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.020418301 (101h51m46.482510824s) [noise=true , address=145.79.227.7:4293 , piece_id=42VM53KR7S7HWHIVQPUUNNRFX5OVT4GBN3BKKQOFH7UJIBJHQN4A, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.022297389 (101h51m46.484389872s) [noise=true , address=45.91.116.5:4299 , piece_id=FZMU5DZZYREA4MXMLZIH6PXUDAKSSH7MZVD35SOQ7TPARF6MT5PQ, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.030930067 (101h51m46.493022561s) [noise=true , address=145.79.227.3:4149 , piece_id=CGXWXBPPQVOCVFVSQ576G3CQXNKHK7FI2776IEKKACUSRQREOMOA, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:01.031244316 (101h51m46.493336809s) [noise=true , address=145.79.227.16:4912 , piece_id=BFHV5JIFMCTMMMXVIWIWQIDNLAGWC2LAWHDCYCAFF3WKHRTFIJPQ, seg_pos={0 0}, upload=89637]: Uploading piece... Sep 27 19:23:02.765616825 (101h51m48.227709298s) [upload=89637]: Next returned nil segment Sep 27 19:23:02.765707788 (101h51m48.227800271s) [upload=89637]: Waiting for error group managing segments... Sep 27 19:23:03.050942347 (101h51m48.51303483s) [noise=true , address=145.79.227.17:4972 , piece_id=WDKM3UXNVR374D3CUSPCN23G332LYP6HBZADBW2D73WOTKRCXU7Q, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:03.066525011 (101h51m48.528617494s) [noise=true , address=145.79.227.16:4912 , piece_id=BFHV5JIFMCTMMMXVIWIWQIDNLAGWC2LAWHDCYCAFF3WKHRTFIJPQ, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:03.069000955 (101h51m48.531093428s) [noise=true , address=145.79.227.12:4765 , piece_id=VRNQHMIEDOTON4SPDVBBJ3HQTPCSAOGV435XWJ35RPD6RF275RTQ, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:03.074882284 (101h51m48.536974767s) [noise=true , address=145.79.227.16:4911 , piece_id=PWUJSPH2LNCIEMYULINGYKNJS6K4VL57FL26Z6ROT6RAXOJBWKDA, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:03.080427743 (101h51m48.542520226s) [noise=true , address=145.79.227.13:4804 , piece_id=DMXTP2YNQUF5JMO4GWQHYGRYRXBSYS3EIXHX5OJIXCRA6FCMT6WA, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:03.085761279 (101h51m48.547853762s) [noise=true , address=145.79.227.3:4149 , piece_id=CGXWXBPPQVOCVFVSQ576G3CQXNKHK7FI2776IEKKACUSRQREOMOA, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:03.106344614 (101h51m48.568437097s) [noise=true , address=145.79.227.18:4007 , piece_id=VA4U55INCQJXY733KXRADUJHTDF3FIXGYKLFF6VJCIJVFV5FBKFQ, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:03.186240711 (101h51m48.648333194s) [noise=true , address=77.90.31.15:4697 , piece_id=RYZ5IY6FWLNNRLF2CLJ7RYJOSONBTEIBVBVNFM5ASOTSJ57LFDNQ, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:03.284507278 (101h51m48.746599752s) [noise=true , address=77.90.31.18:4807 , piece_id=GFNGQAB7Z3F4DEZCNIVIIED5S3QDTQU5G5CTWKTTDKO2TT3XTFSQ, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:03.304107621 (101h51m48.766200104s) [noise=true , address=77.90.31.19:4843 , piece_id=LJXNSV6TCVQF4FEASDLBJG7UHGQ5J3F2AOILPQ2PTPMERQT34ECA, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:04.442365286 (101h51m49.90445777s) [noise=true , address=145.79.227.7:4293 , piece_id=42VM53KR7S7HWHIVQPUUNNRFX5OVT4GBN3BKKQOFH7UJIBJHQN4A, seg_pos={0 0}, upload=89637]: Done uploading piece. err: <nil> Sep 27 19:23:29.984729682 (101h52m15.446822155s) [noise=true , address=45.91.116.10:4082 , piece_id=M7D7MIJSPQCBXAHADPH6SPDODVJSLRDCTW5NI5SCPHHATV2BNA4A, seg_pos={0 0}, upload=89637]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: rpc: dial tcp 45.91.116.10:4082: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:23:29.984885788 (101h52m15.446978261s) [noise=true , address=45.91.116.12:4225 , piece_id=XI4P63YFEIFS3EZROLQDQUM2GD522FFKABLJ4PGOFY2QKWRF4WZQ, seg_pos={0 0}, upload=89637]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:23:29.985001008 (101h52m15.447093481s) [noise=true , address=45.91.116.12:4226 , piece_id=FRXDV3XG7K5UVQ3AFS2CKHNJK36VCF3JD6G6PK7SLEU4P7VCFOTA, seg_pos={0 0}, upload=89637]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:23:29.985137719 (101h52m15.447230212s) [noise=true , address=45.91.116.5:4299 , piece_id=FZMU5DZZYREA4MXMLZIH6PXUDAKSSH7MZVD35SOQ7TPARF6MT5PQ, seg_pos={0 0}, upload=89637]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:23:29.985432991 (101h52m15.447525474s) [noise=true , address=45.91.116.9:4262 , piece_id=TBA6RMCZS67TSX555MURJJMTLLKLG43VJ6RXNTAVDR6YD73GEJ3Q, seg_pos={0 0}, upload=89637]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 188.166.225.170:35588->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 19:23:29.985464801 (101h52m15.447557284s) [seg_pos={0 0}, upload=89637]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 27 19:23:30.235762735 (101h52m15.697855209s) [upload=89637]: Done uploading segments... Sep 27 19:23:30.235800337 (101h52m15.69789282s) [upload=89637]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.252752612s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.062616505s FAIL (2025-09-27 19:22:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== Sep 27 19:19:52.443476361 (101h48m37.905568834s) [upload=89589]: Starting upload Sep 27 19:19:52.443561071 (101h48m37.905653545s) [upload=89589]: Done starting upload Sep 27 19:19:52.443645672 (101h48m37.905738145s) [upload=89589]: Uploading segments... Sep 27 19:19:52.443779136 (101h48m37.905871609s) [upload=89589]: Got next segment. Inline: false Sep 27 19:19:52.443814784 (101h48m37.905907267s) [upload=89589]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:19:52.443820506 (101h48m37.905912979s) [upload=89589]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:19:52.690169349 (101h48m38.152261842s) [seg_pos={0 0}, upload=89589]: Begin upload segment... Sep 27 19:19:52.690245113 (101h48m38.152337596s) [seg_pos={0 0}, upload=89589]: Done begin upload segment. Sep 27 19:19:52.690281663 (101h48m38.152374146s) [noise=true , address=45.91.116.12:4225 , piece_id=EJYLBMGANJWRNH5XCZ5DE6IT5QEUPOBOR3LNL7KPGITATG3OV7PQ, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.690334854 (101h48m38.152427337s) [noise=true , address=145.79.227.17:4972 , piece_id=475AW3TOYS7YKQT3NUT4LZEUEANOV2RFKZXUYMLWXLCS7MGIQGRA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.690942972 (101h48m38.153035455s) [noise=true , address=145.79.227.17:4970 , piece_id=LCXJ3HHD352BGVKBNV44SBDB76V3EGA5AYOEGG2S63F2GWWN6Q4A, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.691541591 (101h48m38.153634065s) [noise=true , address=145.79.227.4:4186 , piece_id=R4KDZI2P5HSD44V3YBFJVFGAWMMVPPKKVLNTKQSJTP7HZKABOHMA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.691977843 (101h48m38.154070346s) [noise=true , address=45.91.116.8:4045 , piece_id=NCKXF5ZK4PRM7LTTMDWUHCYZJKDEKMMRO5PJKFNEPE7PGUW6OXXA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.692539873 (101h48m38.154632356s) [noise=true , address=77.90.31.18:4805 , piece_id=5PKRAL425FQWJ2D65VVILB3QCFKRATX7KM7EMBURASTV77CKGKJA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.694741234 (101h48m38.156833717s) [noise=true , address=45.91.116.11:4118 , piece_id=5C4EVONJ3IA4FCHG7TM6TKOENP4SGT6SZUSVQ32CDWJC4CKC3TCA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.697068055 (101h48m38.159160538s) [noise=true , address=45.91.116.14:4191 , piece_id=AUKBJIDXX6OQZJMRVH7K32SF6SAGSBRJEJ53YVHUGTFWCSFXO3NA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.699412810 (101h48m38.161505293s) [noise=true , address=145.79.227.2:4114 , piece_id=EUJSGFULX2S4ZYXTUAML3RRL55G73GAEFRWL36DZIXON7V4KMZ7A, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.700974273 (101h48m38.163066756s) [noise=true , address=45.91.116.10:4081 , piece_id=W4USXIFFW3ZU63ESEANFM5YGTLBRC4TZPFNNQZXTFCFMVLWEWHRA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.702413845 (101h48m38.164506328s) [noise=true , address=45.91.116.7:4010 , piece_id=OOUW65B5LIPWBCJBT5PP4RY2GVROTI4CLI75CO6ORKXKQN5YSC2Q, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.703630942 (101h48m38.165723425s) [noise=true , address=77.90.31.19:4841 , piece_id=RRGPWVCDMWCIPWOUVLBYSQ54D67WCCXRR5KR6TEIG3NDCQHPEYCQ, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.704956987 (101h48m38.16704947s) [noise=true , address=45.91.116.6:4974 , piece_id=X5P5FOHEZLQRE5IOHPTUE2764JENKQUYORLLFBRGGYIX4VVKIHPQ, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.705764435 (101h48m38.167856908s) [noise=true , address=145.79.227.6:4259 , piece_id=R73AII4IS2AYWIHJN257XPRMWXWZOSLM4I46WYCN6ENACT627FIQ, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.707539334 (101h48m38.169631817s) [noise=true , address=145.79.227.14:4839 , piece_id=QQY25UVD27Z3PRN6CG2NPVAISZX6GXOJGERIAJDQ56I4SFGHRAOA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.707900924 (101h48m38.169993407s) [noise=true , address=45.91.116.5:4298 , piece_id=WGDO3ZHDIPIEKZJXOU375GJEEEZQLPZURDIJY6IYDAPCP5XV2FNQ, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:54.439960117 (101h48m39.90205259s) [upload=89589]: Next returned nil segment Sep 27 19:19:54.439979073 (101h48m39.902071556s) [upload=89589]: Waiting for error group managing segments... Sep 27 19:19:54.732048959 (101h48m40.194141442s) [noise=true , address=145.79.227.6:4259 , piece_id=R73AII4IS2AYWIHJN257XPRMWXWZOSLM4I46WYCN6ENACT627FIQ, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.732903857 (101h48m40.19499634s) [noise=true , address=145.79.227.17:4972 , piece_id=475AW3TOYS7YKQT3NUT4LZEUEANOV2RFKZXUYMLWXLCS7MGIQGRA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.736893313 (101h48m40.198985796s) [noise=true , address=145.79.227.2:4114 , piece_id=EUJSGFULX2S4ZYXTUAML3RRL55G73GAEFRWL36DZIXON7V4KMZ7A, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.757789724 (101h48m40.219882197s) [noise=true , address=145.79.227.14:4839 , piece_id=QQY25UVD27Z3PRN6CG2NPVAISZX6GXOJGERIAJDQ56I4SFGHRAOA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.758278014 (101h48m40.220370487s) [noise=true , address=145.79.227.17:4970 , piece_id=LCXJ3HHD352BGVKBNV44SBDB76V3EGA5AYOEGG2S63F2GWWN6Q4A, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.863591040 (101h48m40.325683513s) [noise=true , address=45.91.116.8:4045 , piece_id=NCKXF5ZK4PRM7LTTMDWUHCYZJKDEKMMRO5PJKFNEPE7PGUW6OXXA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.880233069 (101h48m40.342325542s) [noise=true , address=45.91.116.14:4191 , piece_id=AUKBJIDXX6OQZJMRVH7K32SF6SAGSBRJEJ53YVHUGTFWCSFXO3NA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.882815256 (101h48m40.344907729s) [noise=true , address=77.90.31.19:4841 , piece_id=RRGPWVCDMWCIPWOUVLBYSQ54D67WCCXRR5KR6TEIG3NDCQHPEYCQ, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.908663606 (101h48m40.370756089s) [noise=true , address=45.91.116.11:4118 , piece_id=5C4EVONJ3IA4FCHG7TM6TKOENP4SGT6SZUSVQ32CDWJC4CKC3TCA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.933330356 (101h48m40.395422829s) [noise=true , address=45.91.116.5:4298 , piece_id=WGDO3ZHDIPIEKZJXOU375GJEEEZQLPZURDIJY6IYDAPCP5XV2FNQ, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.939175867 (101h48m40.40126834s) [noise=true , address=45.91.116.7:4010 , piece_id=OOUW65B5LIPWBCJBT5PP4RY2GVROTI4CLI75CO6ORKXKQN5YSC2Q, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.964123512 (101h48m40.426215995s) [noise=true , address=77.90.31.18:4805 , piece_id=5PKRAL425FQWJ2D65VVILB3QCFKRATX7KM7EMBURASTV77CKGKJA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.974510210 (101h48m40.436602694s) [noise=true , address=45.91.116.6:4974 , piece_id=X5P5FOHEZLQRE5IOHPTUE2764JENKQUYORLLFBRGGYIX4VVKIHPQ, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.974548233 (101h48m40.436640706s) [seg_pos={0 0}, upload=89589]: Segment reached optimal threshold of 13 pieces. Sep 27 19:19:54.974708317 (101h48m40.4368008s) [noise=true , address=45.91.116.12:4225 , piece_id=EJYLBMGANJWRNH5XCZ5DE6IT5QEUPOBOR3LNL7KPGITATG3OV7PQ, seg_pos={0 0}, upload=89589]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: expected piece hash; context canceled; EOF Sep 27 19:19:54.974718607 (101h48m40.43681108s) [noise=true , address=45.91.116.12:4225 , piece_id=EJYLBMGANJWRNH5XCZ5DE6IT5QEUPOBOR3LNL7KPGITATG3OV7PQ, seg_pos={0 0}, upload=89589]: Long tail cancellation due to optimal threshold Sep 27 19:19:54.974801585 (101h48m40.436894058s) [noise=true , address=145.79.227.4:4186 , piece_id=R4KDZI2P5HSD44V3YBFJVFGAWMMVPPKKVLNTKQSJTP7HZKABOHMA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: write tcp 188.166.225.170:33812->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:33812->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33812->145.79.227.4:4186: use of closed network connection Sep 27 19:19:54.974821864 (101h48m40.436914337s) [noise=true , address=145.79.227.4:4186 , piece_id=R4KDZI2P5HSD44V3YBFJVFGAWMMVPPKKVLNTKQSJTP7HZKABOHMA, seg_pos={0 0}, upload=89589]: Long tail cancellation due to optimal threshold Sep 27 19:19:54.974830510 (101h48m40.436922993s) [noise=true , address=45.91.116.10:4081 , piece_id=W4USXIFFW3ZU63ESEANFM5YGTLBRC4TZPFNNQZXTFCFMVLWEWHRA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 27 19:19:54.974839988 (101h48m40.436932461s) [noise=true , address=45.91.116.10:4081 , piece_id=W4USXIFFW3ZU63ESEANFM5YGTLBRC4TZPFNNQZXTFCFMVLWEWHRA, seg_pos={0 0}, upload=89589]: Long tail cancellation due to optimal threshold Sep 27 19:19:54.974882449 (101h48m40.436974922s) [seg_pos={0 0}, upload=89589]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 19:19:54.974976498 (101h48m40.437068971s) [upload=89589]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 19:19:54.974981578 (101h48m40.437074051s) [upload=89589]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 19:19:55.287167387 (101h48m40.74925986s) [upload=89589]: Done uploading segments... Sep 27 19:19:55.287193477 (101h48m40.749286191s) [upload=89589]: Upload finished. err: <nil> Duration: 6.062641873s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 14.193454572s FAIL (2025-09-27 19:21:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== Sep 27 19:19:52.443476361 (101h48m37.905568834s) [upload=89589]: Starting upload Sep 27 19:19:52.443561071 (101h48m37.905653545s) [upload=89589]: Done starting upload Sep 27 19:19:52.443645672 (101h48m37.905738145s) [upload=89589]: Uploading segments... Sep 27 19:19:52.443779136 (101h48m37.905871609s) [upload=89589]: Got next segment. Inline: false Sep 27 19:19:52.443814784 (101h48m37.905907267s) [upload=89589]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:19:52.443820506 (101h48m37.905912979s) [upload=89589]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:19:52.690169349 (101h48m38.152261842s) [seg_pos={0 0}, upload=89589]: Begin upload segment... Sep 27 19:19:52.690245113 (101h48m38.152337596s) [seg_pos={0 0}, upload=89589]: Done begin upload segment. Sep 27 19:19:52.690281663 (101h48m38.152374146s) [noise=true , address=45.91.116.12:4225 , piece_id=EJYLBMGANJWRNH5XCZ5DE6IT5QEUPOBOR3LNL7KPGITATG3OV7PQ, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.690334854 (101h48m38.152427337s) [noise=true , address=145.79.227.17:4972 , piece_id=475AW3TOYS7YKQT3NUT4LZEUEANOV2RFKZXUYMLWXLCS7MGIQGRA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.690942972 (101h48m38.153035455s) [noise=true , address=145.79.227.17:4970 , piece_id=LCXJ3HHD352BGVKBNV44SBDB76V3EGA5AYOEGG2S63F2GWWN6Q4A, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.691541591 (101h48m38.153634065s) [noise=true , address=145.79.227.4:4186 , piece_id=R4KDZI2P5HSD44V3YBFJVFGAWMMVPPKKVLNTKQSJTP7HZKABOHMA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.691977843 (101h48m38.154070346s) [noise=true , address=45.91.116.8:4045 , piece_id=NCKXF5ZK4PRM7LTTMDWUHCYZJKDEKMMRO5PJKFNEPE7PGUW6OXXA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.692539873 (101h48m38.154632356s) [noise=true , address=77.90.31.18:4805 , piece_id=5PKRAL425FQWJ2D65VVILB3QCFKRATX7KM7EMBURASTV77CKGKJA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.694741234 (101h48m38.156833717s) [noise=true , address=45.91.116.11:4118 , piece_id=5C4EVONJ3IA4FCHG7TM6TKOENP4SGT6SZUSVQ32CDWJC4CKC3TCA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.697068055 (101h48m38.159160538s) [noise=true , address=45.91.116.14:4191 , piece_id=AUKBJIDXX6OQZJMRVH7K32SF6SAGSBRJEJ53YVHUGTFWCSFXO3NA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.699412810 (101h48m38.161505293s) [noise=true , address=145.79.227.2:4114 , piece_id=EUJSGFULX2S4ZYXTUAML3RRL55G73GAEFRWL36DZIXON7V4KMZ7A, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.700974273 (101h48m38.163066756s) [noise=true , address=45.91.116.10:4081 , piece_id=W4USXIFFW3ZU63ESEANFM5YGTLBRC4TZPFNNQZXTFCFMVLWEWHRA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.702413845 (101h48m38.164506328s) [noise=true , address=45.91.116.7:4010 , piece_id=OOUW65B5LIPWBCJBT5PP4RY2GVROTI4CLI75CO6ORKXKQN5YSC2Q, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.703630942 (101h48m38.165723425s) [noise=true , address=77.90.31.19:4841 , piece_id=RRGPWVCDMWCIPWOUVLBYSQ54D67WCCXRR5KR6TEIG3NDCQHPEYCQ, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.704956987 (101h48m38.16704947s) [noise=true , address=45.91.116.6:4974 , piece_id=X5P5FOHEZLQRE5IOHPTUE2764JENKQUYORLLFBRGGYIX4VVKIHPQ, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.705764435 (101h48m38.167856908s) [noise=true , address=145.79.227.6:4259 , piece_id=R73AII4IS2AYWIHJN257XPRMWXWZOSLM4I46WYCN6ENACT627FIQ, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.707539334 (101h48m38.169631817s) [noise=true , address=145.79.227.14:4839 , piece_id=QQY25UVD27Z3PRN6CG2NPVAISZX6GXOJGERIAJDQ56I4SFGHRAOA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.707900924 (101h48m38.169993407s) [noise=true , address=45.91.116.5:4298 , piece_id=WGDO3ZHDIPIEKZJXOU375GJEEEZQLPZURDIJY6IYDAPCP5XV2FNQ, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:54.439960117 (101h48m39.90205259s) [upload=89589]: Next returned nil segment Sep 27 19:19:54.439979073 (101h48m39.902071556s) [upload=89589]: Waiting for error group managing segments... Sep 27 19:19:54.732048959 (101h48m40.194141442s) [noise=true , address=145.79.227.6:4259 , piece_id=R73AII4IS2AYWIHJN257XPRMWXWZOSLM4I46WYCN6ENACT627FIQ, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.732903857 (101h48m40.19499634s) [noise=true , address=145.79.227.17:4972 , piece_id=475AW3TOYS7YKQT3NUT4LZEUEANOV2RFKZXUYMLWXLCS7MGIQGRA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.736893313 (101h48m40.198985796s) [noise=true , address=145.79.227.2:4114 , piece_id=EUJSGFULX2S4ZYXTUAML3RRL55G73GAEFRWL36DZIXON7V4KMZ7A, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.757789724 (101h48m40.219882197s) [noise=true , address=145.79.227.14:4839 , piece_id=QQY25UVD27Z3PRN6CG2NPVAISZX6GXOJGERIAJDQ56I4SFGHRAOA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.758278014 (101h48m40.220370487s) [noise=true , address=145.79.227.17:4970 , piece_id=LCXJ3HHD352BGVKBNV44SBDB76V3EGA5AYOEGG2S63F2GWWN6Q4A, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.863591040 (101h48m40.325683513s) [noise=true , address=45.91.116.8:4045 , piece_id=NCKXF5ZK4PRM7LTTMDWUHCYZJKDEKMMRO5PJKFNEPE7PGUW6OXXA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.880233069 (101h48m40.342325542s) [noise=true , address=45.91.116.14:4191 , piece_id=AUKBJIDXX6OQZJMRVH7K32SF6SAGSBRJEJ53YVHUGTFWCSFXO3NA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.882815256 (101h48m40.344907729s) [noise=true , address=77.90.31.19:4841 , piece_id=RRGPWVCDMWCIPWOUVLBYSQ54D67WCCXRR5KR6TEIG3NDCQHPEYCQ, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.908663606 (101h48m40.370756089s) [noise=true , address=45.91.116.11:4118 , piece_id=5C4EVONJ3IA4FCHG7TM6TKOENP4SGT6SZUSVQ32CDWJC4CKC3TCA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.933330356 (101h48m40.395422829s) [noise=true , address=45.91.116.5:4298 , piece_id=WGDO3ZHDIPIEKZJXOU375GJEEEZQLPZURDIJY6IYDAPCP5XV2FNQ, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.939175867 (101h48m40.40126834s) [noise=true , address=45.91.116.7:4010 , piece_id=OOUW65B5LIPWBCJBT5PP4RY2GVROTI4CLI75CO6ORKXKQN5YSC2Q, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.964123512 (101h48m40.426215995s) [noise=true , address=77.90.31.18:4805 , piece_id=5PKRAL425FQWJ2D65VVILB3QCFKRATX7KM7EMBURASTV77CKGKJA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.974510210 (101h48m40.436602694s) [noise=true , address=45.91.116.6:4974 , piece_id=X5P5FOHEZLQRE5IOHPTUE2764JENKQUYORLLFBRGGYIX4VVKIHPQ, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.974548233 (101h48m40.436640706s) [seg_pos={0 0}, upload=89589]: Segment reached optimal threshold of 13 pieces. Sep 27 19:19:54.974708317 (101h48m40.4368008s) [noise=true , address=45.91.116.12:4225 , piece_id=EJYLBMGANJWRNH5XCZ5DE6IT5QEUPOBOR3LNL7KPGITATG3OV7PQ, seg_pos={0 0}, upload=89589]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: expected piece hash; context canceled; EOF Sep 27 19:19:54.974718607 (101h48m40.43681108s) [noise=true , address=45.91.116.12:4225 , piece_id=EJYLBMGANJWRNH5XCZ5DE6IT5QEUPOBOR3LNL7KPGITATG3OV7PQ, seg_pos={0 0}, upload=89589]: Long tail cancellation due to optimal threshold Sep 27 19:19:54.974801585 (101h48m40.436894058s) [noise=true , address=145.79.227.4:4186 , piece_id=R4KDZI2P5HSD44V3YBFJVFGAWMMVPPKKVLNTKQSJTP7HZKABOHMA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: write tcp 188.166.225.170:33812->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:33812->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33812->145.79.227.4:4186: use of closed network connection Sep 27 19:19:54.974821864 (101h48m40.436914337s) [noise=true , address=145.79.227.4:4186 , piece_id=R4KDZI2P5HSD44V3YBFJVFGAWMMVPPKKVLNTKQSJTP7HZKABOHMA, seg_pos={0 0}, upload=89589]: Long tail cancellation due to optimal threshold Sep 27 19:19:54.974830510 (101h48m40.436922993s) [noise=true , address=45.91.116.10:4081 , piece_id=W4USXIFFW3ZU63ESEANFM5YGTLBRC4TZPFNNQZXTFCFMVLWEWHRA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 27 19:19:54.974839988 (101h48m40.436932461s) [noise=true , address=45.91.116.10:4081 , piece_id=W4USXIFFW3ZU63ESEANFM5YGTLBRC4TZPFNNQZXTFCFMVLWEWHRA, seg_pos={0 0}, upload=89589]: Long tail cancellation due to optimal threshold Sep 27 19:19:54.974882449 (101h48m40.436974922s) [seg_pos={0 0}, upload=89589]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 19:19:54.974976498 (101h48m40.437068971s) [upload=89589]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 19:19:54.974981578 (101h48m40.437074051s) [upload=89589]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 19:19:55.287167387 (101h48m40.74925986s) [upload=89589]: Done uploading segments... Sep 27 19:19:55.287193477 (101h48m40.749286191s) [upload=89589]: Upload finished. err: <nil> Duration: 14.193489659s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.215371297s FAIL (2025-09-27 19:20:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== Sep 27 19:19:52.443476361 (101h48m37.905568834s) [upload=89589]: Starting upload Sep 27 19:19:52.443561071 (101h48m37.905653545s) [upload=89589]: Done starting upload Sep 27 19:19:52.443645672 (101h48m37.905738145s) [upload=89589]: Uploading segments... Sep 27 19:19:52.443779136 (101h48m37.905871609s) [upload=89589]: Got next segment. Inline: false Sep 27 19:19:52.443814784 (101h48m37.905907267s) [upload=89589]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:19:52.443820506 (101h48m37.905912979s) [upload=89589]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:19:52.690169349 (101h48m38.152261842s) [seg_pos={0 0}, upload=89589]: Begin upload segment... Sep 27 19:19:52.690245113 (101h48m38.152337596s) [seg_pos={0 0}, upload=89589]: Done begin upload segment. Sep 27 19:19:52.690281663 (101h48m38.152374146s) [noise=true , address=45.91.116.12:4225 , piece_id=EJYLBMGANJWRNH5XCZ5DE6IT5QEUPOBOR3LNL7KPGITATG3OV7PQ, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.690334854 (101h48m38.152427337s) [noise=true , address=145.79.227.17:4972 , piece_id=475AW3TOYS7YKQT3NUT4LZEUEANOV2RFKZXUYMLWXLCS7MGIQGRA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.690942972 (101h48m38.153035455s) [noise=true , address=145.79.227.17:4970 , piece_id=LCXJ3HHD352BGVKBNV44SBDB76V3EGA5AYOEGG2S63F2GWWN6Q4A, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.691541591 (101h48m38.153634065s) [noise=true , address=145.79.227.4:4186 , piece_id=R4KDZI2P5HSD44V3YBFJVFGAWMMVPPKKVLNTKQSJTP7HZKABOHMA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.691977843 (101h48m38.154070346s) [noise=true , address=45.91.116.8:4045 , piece_id=NCKXF5ZK4PRM7LTTMDWUHCYZJKDEKMMRO5PJKFNEPE7PGUW6OXXA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.692539873 (101h48m38.154632356s) [noise=true , address=77.90.31.18:4805 , piece_id=5PKRAL425FQWJ2D65VVILB3QCFKRATX7KM7EMBURASTV77CKGKJA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.694741234 (101h48m38.156833717s) [noise=true , address=45.91.116.11:4118 , piece_id=5C4EVONJ3IA4FCHG7TM6TKOENP4SGT6SZUSVQ32CDWJC4CKC3TCA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.697068055 (101h48m38.159160538s) [noise=true , address=45.91.116.14:4191 , piece_id=AUKBJIDXX6OQZJMRVH7K32SF6SAGSBRJEJ53YVHUGTFWCSFXO3NA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.699412810 (101h48m38.161505293s) [noise=true , address=145.79.227.2:4114 , piece_id=EUJSGFULX2S4ZYXTUAML3RRL55G73GAEFRWL36DZIXON7V4KMZ7A, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.700974273 (101h48m38.163066756s) [noise=true , address=45.91.116.10:4081 , piece_id=W4USXIFFW3ZU63ESEANFM5YGTLBRC4TZPFNNQZXTFCFMVLWEWHRA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.702413845 (101h48m38.164506328s) [noise=true , address=45.91.116.7:4010 , piece_id=OOUW65B5LIPWBCJBT5PP4RY2GVROTI4CLI75CO6ORKXKQN5YSC2Q, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.703630942 (101h48m38.165723425s) [noise=true , address=77.90.31.19:4841 , piece_id=RRGPWVCDMWCIPWOUVLBYSQ54D67WCCXRR5KR6TEIG3NDCQHPEYCQ, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.704956987 (101h48m38.16704947s) [noise=true , address=45.91.116.6:4974 , piece_id=X5P5FOHEZLQRE5IOHPTUE2764JENKQUYORLLFBRGGYIX4VVKIHPQ, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.705764435 (101h48m38.167856908s) [noise=true , address=145.79.227.6:4259 , piece_id=R73AII4IS2AYWIHJN257XPRMWXWZOSLM4I46WYCN6ENACT627FIQ, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.707539334 (101h48m38.169631817s) [noise=true , address=145.79.227.14:4839 , piece_id=QQY25UVD27Z3PRN6CG2NPVAISZX6GXOJGERIAJDQ56I4SFGHRAOA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.707900924 (101h48m38.169993407s) [noise=true , address=45.91.116.5:4298 , piece_id=WGDO3ZHDIPIEKZJXOU375GJEEEZQLPZURDIJY6IYDAPCP5XV2FNQ, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:54.439960117 (101h48m39.90205259s) [upload=89589]: Next returned nil segment Sep 27 19:19:54.439979073 (101h48m39.902071556s) [upload=89589]: Waiting for error group managing segments... Sep 27 19:19:54.732048959 (101h48m40.194141442s) [noise=true , address=145.79.227.6:4259 , piece_id=R73AII4IS2AYWIHJN257XPRMWXWZOSLM4I46WYCN6ENACT627FIQ, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.732903857 (101h48m40.19499634s) [noise=true , address=145.79.227.17:4972 , piece_id=475AW3TOYS7YKQT3NUT4LZEUEANOV2RFKZXUYMLWXLCS7MGIQGRA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.736893313 (101h48m40.198985796s) [noise=true , address=145.79.227.2:4114 , piece_id=EUJSGFULX2S4ZYXTUAML3RRL55G73GAEFRWL36DZIXON7V4KMZ7A, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.757789724 (101h48m40.219882197s) [noise=true , address=145.79.227.14:4839 , piece_id=QQY25UVD27Z3PRN6CG2NPVAISZX6GXOJGERIAJDQ56I4SFGHRAOA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.758278014 (101h48m40.220370487s) [noise=true , address=145.79.227.17:4970 , piece_id=LCXJ3HHD352BGVKBNV44SBDB76V3EGA5AYOEGG2S63F2GWWN6Q4A, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.863591040 (101h48m40.325683513s) [noise=true , address=45.91.116.8:4045 , piece_id=NCKXF5ZK4PRM7LTTMDWUHCYZJKDEKMMRO5PJKFNEPE7PGUW6OXXA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.880233069 (101h48m40.342325542s) [noise=true , address=45.91.116.14:4191 , piece_id=AUKBJIDXX6OQZJMRVH7K32SF6SAGSBRJEJ53YVHUGTFWCSFXO3NA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.882815256 (101h48m40.344907729s) [noise=true , address=77.90.31.19:4841 , piece_id=RRGPWVCDMWCIPWOUVLBYSQ54D67WCCXRR5KR6TEIG3NDCQHPEYCQ, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.908663606 (101h48m40.370756089s) [noise=true , address=45.91.116.11:4118 , piece_id=5C4EVONJ3IA4FCHG7TM6TKOENP4SGT6SZUSVQ32CDWJC4CKC3TCA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.933330356 (101h48m40.395422829s) [noise=true , address=45.91.116.5:4298 , piece_id=WGDO3ZHDIPIEKZJXOU375GJEEEZQLPZURDIJY6IYDAPCP5XV2FNQ, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.939175867 (101h48m40.40126834s) [noise=true , address=45.91.116.7:4010 , piece_id=OOUW65B5LIPWBCJBT5PP4RY2GVROTI4CLI75CO6ORKXKQN5YSC2Q, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.964123512 (101h48m40.426215995s) [noise=true , address=77.90.31.18:4805 , piece_id=5PKRAL425FQWJ2D65VVILB3QCFKRATX7KM7EMBURASTV77CKGKJA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.974510210 (101h48m40.436602694s) [noise=true , address=45.91.116.6:4974 , piece_id=X5P5FOHEZLQRE5IOHPTUE2764JENKQUYORLLFBRGGYIX4VVKIHPQ, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.974548233 (101h48m40.436640706s) [seg_pos={0 0}, upload=89589]: Segment reached optimal threshold of 13 pieces. Sep 27 19:19:54.974708317 (101h48m40.4368008s) [noise=true , address=45.91.116.12:4225 , piece_id=EJYLBMGANJWRNH5XCZ5DE6IT5QEUPOBOR3LNL7KPGITATG3OV7PQ, seg_pos={0 0}, upload=89589]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: expected piece hash; context canceled; EOF Sep 27 19:19:54.974718607 (101h48m40.43681108s) [noise=true , address=45.91.116.12:4225 , piece_id=EJYLBMGANJWRNH5XCZ5DE6IT5QEUPOBOR3LNL7KPGITATG3OV7PQ, seg_pos={0 0}, upload=89589]: Long tail cancellation due to optimal threshold Sep 27 19:19:54.974801585 (101h48m40.436894058s) [noise=true , address=145.79.227.4:4186 , piece_id=R4KDZI2P5HSD44V3YBFJVFGAWMMVPPKKVLNTKQSJTP7HZKABOHMA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: write tcp 188.166.225.170:33812->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:33812->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33812->145.79.227.4:4186: use of closed network connection Sep 27 19:19:54.974821864 (101h48m40.436914337s) [noise=true , address=145.79.227.4:4186 , piece_id=R4KDZI2P5HSD44V3YBFJVFGAWMMVPPKKVLNTKQSJTP7HZKABOHMA, seg_pos={0 0}, upload=89589]: Long tail cancellation due to optimal threshold Sep 27 19:19:54.974830510 (101h48m40.436922993s) [noise=true , address=45.91.116.10:4081 , piece_id=W4USXIFFW3ZU63ESEANFM5YGTLBRC4TZPFNNQZXTFCFMVLWEWHRA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 27 19:19:54.974839988 (101h48m40.436932461s) [noise=true , address=45.91.116.10:4081 , piece_id=W4USXIFFW3ZU63ESEANFM5YGTLBRC4TZPFNNQZXTFCFMVLWEWHRA, seg_pos={0 0}, upload=89589]: Long tail cancellation due to optimal threshold Sep 27 19:19:54.974882449 (101h48m40.436974922s) [seg_pos={0 0}, upload=89589]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 19:19:54.974976498 (101h48m40.437068971s) [upload=89589]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 19:19:54.974981578 (101h48m40.437074051s) [upload=89589]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 19:19:55.287167387 (101h48m40.74925986s) [upload=89589]: Done uploading segments... Sep 27 19:19:55.287193477 (101h48m40.749286191s) [upload=89589]: Upload finished. err: <nil> Duration: 6.215401435s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 21.839458758s FAIL (2025-09-27 19:20:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== Sep 27 19:19:52.443476361 (101h48m37.905568834s) [upload=89589]: Starting upload Sep 27 19:19:52.443561071 (101h48m37.905653545s) [upload=89589]: Done starting upload Sep 27 19:19:52.443645672 (101h48m37.905738145s) [upload=89589]: Uploading segments... Sep 27 19:19:52.443779136 (101h48m37.905871609s) [upload=89589]: Got next segment. Inline: false Sep 27 19:19:52.443814784 (101h48m37.905907267s) [upload=89589]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:19:52.443820506 (101h48m37.905912979s) [upload=89589]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:19:52.690169349 (101h48m38.152261842s) [seg_pos={0 0}, upload=89589]: Begin upload segment... Sep 27 19:19:52.690245113 (101h48m38.152337596s) [seg_pos={0 0}, upload=89589]: Done begin upload segment. Sep 27 19:19:52.690281663 (101h48m38.152374146s) [noise=true , address=45.91.116.12:4225 , piece_id=EJYLBMGANJWRNH5XCZ5DE6IT5QEUPOBOR3LNL7KPGITATG3OV7PQ, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.690334854 (101h48m38.152427337s) [noise=true , address=145.79.227.17:4972 , piece_id=475AW3TOYS7YKQT3NUT4LZEUEANOV2RFKZXUYMLWXLCS7MGIQGRA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.690942972 (101h48m38.153035455s) [noise=true , address=145.79.227.17:4970 , piece_id=LCXJ3HHD352BGVKBNV44SBDB76V3EGA5AYOEGG2S63F2GWWN6Q4A, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.691541591 (101h48m38.153634065s) [noise=true , address=145.79.227.4:4186 , piece_id=R4KDZI2P5HSD44V3YBFJVFGAWMMVPPKKVLNTKQSJTP7HZKABOHMA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.691977843 (101h48m38.154070346s) [noise=true , address=45.91.116.8:4045 , piece_id=NCKXF5ZK4PRM7LTTMDWUHCYZJKDEKMMRO5PJKFNEPE7PGUW6OXXA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.692539873 (101h48m38.154632356s) [noise=true , address=77.90.31.18:4805 , piece_id=5PKRAL425FQWJ2D65VVILB3QCFKRATX7KM7EMBURASTV77CKGKJA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.694741234 (101h48m38.156833717s) [noise=true , address=45.91.116.11:4118 , piece_id=5C4EVONJ3IA4FCHG7TM6TKOENP4SGT6SZUSVQ32CDWJC4CKC3TCA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.697068055 (101h48m38.159160538s) [noise=true , address=45.91.116.14:4191 , piece_id=AUKBJIDXX6OQZJMRVH7K32SF6SAGSBRJEJ53YVHUGTFWCSFXO3NA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.699412810 (101h48m38.161505293s) [noise=true , address=145.79.227.2:4114 , piece_id=EUJSGFULX2S4ZYXTUAML3RRL55G73GAEFRWL36DZIXON7V4KMZ7A, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.700974273 (101h48m38.163066756s) [noise=true , address=45.91.116.10:4081 , piece_id=W4USXIFFW3ZU63ESEANFM5YGTLBRC4TZPFNNQZXTFCFMVLWEWHRA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.702413845 (101h48m38.164506328s) [noise=true , address=45.91.116.7:4010 , piece_id=OOUW65B5LIPWBCJBT5PP4RY2GVROTI4CLI75CO6ORKXKQN5YSC2Q, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.703630942 (101h48m38.165723425s) [noise=true , address=77.90.31.19:4841 , piece_id=RRGPWVCDMWCIPWOUVLBYSQ54D67WCCXRR5KR6TEIG3NDCQHPEYCQ, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.704956987 (101h48m38.16704947s) [noise=true , address=45.91.116.6:4974 , piece_id=X5P5FOHEZLQRE5IOHPTUE2764JENKQUYORLLFBRGGYIX4VVKIHPQ, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.705764435 (101h48m38.167856908s) [noise=true , address=145.79.227.6:4259 , piece_id=R73AII4IS2AYWIHJN257XPRMWXWZOSLM4I46WYCN6ENACT627FIQ, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.707539334 (101h48m38.169631817s) [noise=true , address=145.79.227.14:4839 , piece_id=QQY25UVD27Z3PRN6CG2NPVAISZX6GXOJGERIAJDQ56I4SFGHRAOA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.707900924 (101h48m38.169993407s) [noise=true , address=45.91.116.5:4298 , piece_id=WGDO3ZHDIPIEKZJXOU375GJEEEZQLPZURDIJY6IYDAPCP5XV2FNQ, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:54.439960117 (101h48m39.90205259s) [upload=89589]: Next returned nil segment Sep 27 19:19:54.439979073 (101h48m39.902071556s) [upload=89589]: Waiting for error group managing segments... Sep 27 19:19:54.732048959 (101h48m40.194141442s) [noise=true , address=145.79.227.6:4259 , piece_id=R73AII4IS2AYWIHJN257XPRMWXWZOSLM4I46WYCN6ENACT627FIQ, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.732903857 (101h48m40.19499634s) [noise=true , address=145.79.227.17:4972 , piece_id=475AW3TOYS7YKQT3NUT4LZEUEANOV2RFKZXUYMLWXLCS7MGIQGRA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.736893313 (101h48m40.198985796s) [noise=true , address=145.79.227.2:4114 , piece_id=EUJSGFULX2S4ZYXTUAML3RRL55G73GAEFRWL36DZIXON7V4KMZ7A, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.757789724 (101h48m40.219882197s) [noise=true , address=145.79.227.14:4839 , piece_id=QQY25UVD27Z3PRN6CG2NPVAISZX6GXOJGERIAJDQ56I4SFGHRAOA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.758278014 (101h48m40.220370487s) [noise=true , address=145.79.227.17:4970 , piece_id=LCXJ3HHD352BGVKBNV44SBDB76V3EGA5AYOEGG2S63F2GWWN6Q4A, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.863591040 (101h48m40.325683513s) [noise=true , address=45.91.116.8:4045 , piece_id=NCKXF5ZK4PRM7LTTMDWUHCYZJKDEKMMRO5PJKFNEPE7PGUW6OXXA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.880233069 (101h48m40.342325542s) [noise=true , address=45.91.116.14:4191 , piece_id=AUKBJIDXX6OQZJMRVH7K32SF6SAGSBRJEJ53YVHUGTFWCSFXO3NA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.882815256 (101h48m40.344907729s) [noise=true , address=77.90.31.19:4841 , piece_id=RRGPWVCDMWCIPWOUVLBYSQ54D67WCCXRR5KR6TEIG3NDCQHPEYCQ, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.908663606 (101h48m40.370756089s) [noise=true , address=45.91.116.11:4118 , piece_id=5C4EVONJ3IA4FCHG7TM6TKOENP4SGT6SZUSVQ32CDWJC4CKC3TCA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.933330356 (101h48m40.395422829s) [noise=true , address=45.91.116.5:4298 , piece_id=WGDO3ZHDIPIEKZJXOU375GJEEEZQLPZURDIJY6IYDAPCP5XV2FNQ, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.939175867 (101h48m40.40126834s) [noise=true , address=45.91.116.7:4010 , piece_id=OOUW65B5LIPWBCJBT5PP4RY2GVROTI4CLI75CO6ORKXKQN5YSC2Q, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.964123512 (101h48m40.426215995s) [noise=true , address=77.90.31.18:4805 , piece_id=5PKRAL425FQWJ2D65VVILB3QCFKRATX7KM7EMBURASTV77CKGKJA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.974510210 (101h48m40.436602694s) [noise=true , address=45.91.116.6:4974 , piece_id=X5P5FOHEZLQRE5IOHPTUE2764JENKQUYORLLFBRGGYIX4VVKIHPQ, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.974548233 (101h48m40.436640706s) [seg_pos={0 0}, upload=89589]: Segment reached optimal threshold of 13 pieces. Sep 27 19:19:54.974708317 (101h48m40.4368008s) [noise=true , address=45.91.116.12:4225 , piece_id=EJYLBMGANJWRNH5XCZ5DE6IT5QEUPOBOR3LNL7KPGITATG3OV7PQ, seg_pos={0 0}, upload=89589]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: expected piece hash; context canceled; EOF Sep 27 19:19:54.974718607 (101h48m40.43681108s) [noise=true , address=45.91.116.12:4225 , piece_id=EJYLBMGANJWRNH5XCZ5DE6IT5QEUPOBOR3LNL7KPGITATG3OV7PQ, seg_pos={0 0}, upload=89589]: Long tail cancellation due to optimal threshold Sep 27 19:19:54.974801585 (101h48m40.436894058s) [noise=true , address=145.79.227.4:4186 , piece_id=R4KDZI2P5HSD44V3YBFJVFGAWMMVPPKKVLNTKQSJTP7HZKABOHMA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: write tcp 188.166.225.170:33812->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:33812->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33812->145.79.227.4:4186: use of closed network connection Sep 27 19:19:54.974821864 (101h48m40.436914337s) [noise=true , address=145.79.227.4:4186 , piece_id=R4KDZI2P5HSD44V3YBFJVFGAWMMVPPKKVLNTKQSJTP7HZKABOHMA, seg_pos={0 0}, upload=89589]: Long tail cancellation due to optimal threshold Sep 27 19:19:54.974830510 (101h48m40.436922993s) [noise=true , address=45.91.116.10:4081 , piece_id=W4USXIFFW3ZU63ESEANFM5YGTLBRC4TZPFNNQZXTFCFMVLWEWHRA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 27 19:19:54.974839988 (101h48m40.436932461s) [noise=true , address=45.91.116.10:4081 , piece_id=W4USXIFFW3ZU63ESEANFM5YGTLBRC4TZPFNNQZXTFCFMVLWEWHRA, seg_pos={0 0}, upload=89589]: Long tail cancellation due to optimal threshold Sep 27 19:19:54.974882449 (101h48m40.436974922s) [seg_pos={0 0}, upload=89589]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 19:19:54.974976498 (101h48m40.437068971s) [upload=89589]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 19:19:54.974981578 (101h48m40.437074051s) [upload=89589]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 19:19:55.287167387 (101h48m40.74925986s) [upload=89589]: Done uploading segments... Sep 27 19:19:55.287193477 (101h48m40.749286191s) [upload=89589]: Upload finished. err: <nil> Duration: 21.839493052s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 17.626930084s FAIL (2025-09-27 19:20:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== Sep 27 19:19:52.443476361 (101h48m37.905568834s) [upload=89589]: Starting upload Sep 27 19:19:52.443561071 (101h48m37.905653545s) [upload=89589]: Done starting upload Sep 27 19:19:52.443645672 (101h48m37.905738145s) [upload=89589]: Uploading segments... Sep 27 19:19:52.443779136 (101h48m37.905871609s) [upload=89589]: Got next segment. Inline: false Sep 27 19:19:52.443814784 (101h48m37.905907267s) [upload=89589]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 19:19:52.443820506 (101h48m37.905912979s) [upload=89589]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 19:19:52.690169349 (101h48m38.152261842s) [seg_pos={0 0}, upload=89589]: Begin upload segment... Sep 27 19:19:52.690245113 (101h48m38.152337596s) [seg_pos={0 0}, upload=89589]: Done begin upload segment. Sep 27 19:19:52.690281663 (101h48m38.152374146s) [noise=true , address=45.91.116.12:4225 , piece_id=EJYLBMGANJWRNH5XCZ5DE6IT5QEUPOBOR3LNL7KPGITATG3OV7PQ, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.690334854 (101h48m38.152427337s) [noise=true , address=145.79.227.17:4972 , piece_id=475AW3TOYS7YKQT3NUT4LZEUEANOV2RFKZXUYMLWXLCS7MGIQGRA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.690942972 (101h48m38.153035455s) [noise=true , address=145.79.227.17:4970 , piece_id=LCXJ3HHD352BGVKBNV44SBDB76V3EGA5AYOEGG2S63F2GWWN6Q4A, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.691541591 (101h48m38.153634065s) [noise=true , address=145.79.227.4:4186 , piece_id=R4KDZI2P5HSD44V3YBFJVFGAWMMVPPKKVLNTKQSJTP7HZKABOHMA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.691977843 (101h48m38.154070346s) [noise=true , address=45.91.116.8:4045 , piece_id=NCKXF5ZK4PRM7LTTMDWUHCYZJKDEKMMRO5PJKFNEPE7PGUW6OXXA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.692539873 (101h48m38.154632356s) [noise=true , address=77.90.31.18:4805 , piece_id=5PKRAL425FQWJ2D65VVILB3QCFKRATX7KM7EMBURASTV77CKGKJA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.694741234 (101h48m38.156833717s) [noise=true , address=45.91.116.11:4118 , piece_id=5C4EVONJ3IA4FCHG7TM6TKOENP4SGT6SZUSVQ32CDWJC4CKC3TCA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.697068055 (101h48m38.159160538s) [noise=true , address=45.91.116.14:4191 , piece_id=AUKBJIDXX6OQZJMRVH7K32SF6SAGSBRJEJ53YVHUGTFWCSFXO3NA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.699412810 (101h48m38.161505293s) [noise=true , address=145.79.227.2:4114 , piece_id=EUJSGFULX2S4ZYXTUAML3RRL55G73GAEFRWL36DZIXON7V4KMZ7A, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.700974273 (101h48m38.163066756s) [noise=true , address=45.91.116.10:4081 , piece_id=W4USXIFFW3ZU63ESEANFM5YGTLBRC4TZPFNNQZXTFCFMVLWEWHRA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.702413845 (101h48m38.164506328s) [noise=true , address=45.91.116.7:4010 , piece_id=OOUW65B5LIPWBCJBT5PP4RY2GVROTI4CLI75CO6ORKXKQN5YSC2Q, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.703630942 (101h48m38.165723425s) [noise=true , address=77.90.31.19:4841 , piece_id=RRGPWVCDMWCIPWOUVLBYSQ54D67WCCXRR5KR6TEIG3NDCQHPEYCQ, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.704956987 (101h48m38.16704947s) [noise=true , address=45.91.116.6:4974 , piece_id=X5P5FOHEZLQRE5IOHPTUE2764JENKQUYORLLFBRGGYIX4VVKIHPQ, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.705764435 (101h48m38.167856908s) [noise=true , address=145.79.227.6:4259 , piece_id=R73AII4IS2AYWIHJN257XPRMWXWZOSLM4I46WYCN6ENACT627FIQ, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.707539334 (101h48m38.169631817s) [noise=true , address=145.79.227.14:4839 , piece_id=QQY25UVD27Z3PRN6CG2NPVAISZX6GXOJGERIAJDQ56I4SFGHRAOA, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:52.707900924 (101h48m38.169993407s) [noise=true , address=45.91.116.5:4298 , piece_id=WGDO3ZHDIPIEKZJXOU375GJEEEZQLPZURDIJY6IYDAPCP5XV2FNQ, seg_pos={0 0}, upload=89589]: Uploading piece... Sep 27 19:19:54.439960117 (101h48m39.90205259s) [upload=89589]: Next returned nil segment Sep 27 19:19:54.439979073 (101h48m39.902071556s) [upload=89589]: Waiting for error group managing segments... Sep 27 19:19:54.732048959 (101h48m40.194141442s) [noise=true , address=145.79.227.6:4259 , piece_id=R73AII4IS2AYWIHJN257XPRMWXWZOSLM4I46WYCN6ENACT627FIQ, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.732903857 (101h48m40.19499634s) [noise=true , address=145.79.227.17:4972 , piece_id=475AW3TOYS7YKQT3NUT4LZEUEANOV2RFKZXUYMLWXLCS7MGIQGRA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.736893313 (101h48m40.198985796s) [noise=true , address=145.79.227.2:4114 , piece_id=EUJSGFULX2S4ZYXTUAML3RRL55G73GAEFRWL36DZIXON7V4KMZ7A, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.757789724 (101h48m40.219882197s) [noise=true , address=145.79.227.14:4839 , piece_id=QQY25UVD27Z3PRN6CG2NPVAISZX6GXOJGERIAJDQ56I4SFGHRAOA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.758278014 (101h48m40.220370487s) [noise=true , address=145.79.227.17:4970 , piece_id=LCXJ3HHD352BGVKBNV44SBDB76V3EGA5AYOEGG2S63F2GWWN6Q4A, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.863591040 (101h48m40.325683513s) [noise=true , address=45.91.116.8:4045 , piece_id=NCKXF5ZK4PRM7LTTMDWUHCYZJKDEKMMRO5PJKFNEPE7PGUW6OXXA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.880233069 (101h48m40.342325542s) [noise=true , address=45.91.116.14:4191 , piece_id=AUKBJIDXX6OQZJMRVH7K32SF6SAGSBRJEJ53YVHUGTFWCSFXO3NA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.882815256 (101h48m40.344907729s) [noise=true , address=77.90.31.19:4841 , piece_id=RRGPWVCDMWCIPWOUVLBYSQ54D67WCCXRR5KR6TEIG3NDCQHPEYCQ, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.908663606 (101h48m40.370756089s) [noise=true , address=45.91.116.11:4118 , piece_id=5C4EVONJ3IA4FCHG7TM6TKOENP4SGT6SZUSVQ32CDWJC4CKC3TCA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.933330356 (101h48m40.395422829s) [noise=true , address=45.91.116.5:4298 , piece_id=WGDO3ZHDIPIEKZJXOU375GJEEEZQLPZURDIJY6IYDAPCP5XV2FNQ, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.939175867 (101h48m40.40126834s) [noise=true , address=45.91.116.7:4010 , piece_id=OOUW65B5LIPWBCJBT5PP4RY2GVROTI4CLI75CO6ORKXKQN5YSC2Q, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.964123512 (101h48m40.426215995s) [noise=true , address=77.90.31.18:4805 , piece_id=5PKRAL425FQWJ2D65VVILB3QCFKRATX7KM7EMBURASTV77CKGKJA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.974510210 (101h48m40.436602694s) [noise=true , address=45.91.116.6:4974 , piece_id=X5P5FOHEZLQRE5IOHPTUE2764JENKQUYORLLFBRGGYIX4VVKIHPQ, seg_pos={0 0}, upload=89589]: Done uploading piece. err: <nil> Sep 27 19:19:54.974548233 (101h48m40.436640706s) [seg_pos={0 0}, upload=89589]: Segment reached optimal threshold of 13 pieces. Sep 27 19:19:54.974708317 (101h48m40.4368008s) [noise=true , address=45.91.116.12:4225 , piece_id=EJYLBMGANJWRNH5XCZ5DE6IT5QEUPOBOR3LNL7KPGITATG3OV7PQ, seg_pos={0 0}, upload=89589]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: expected piece hash; context canceled; EOF Sep 27 19:19:54.974718607 (101h48m40.43681108s) [noise=true , address=45.91.116.12:4225 , piece_id=EJYLBMGANJWRNH5XCZ5DE6IT5QEUPOBOR3LNL7KPGITATG3OV7PQ, seg_pos={0 0}, upload=89589]: Long tail cancellation due to optimal threshold Sep 27 19:19:54.974801585 (101h48m40.436894058s) [noise=true , address=145.79.227.4:4186 , piece_id=R4KDZI2P5HSD44V3YBFJVFGAWMMVPPKKVLNTKQSJTP7HZKABOHMA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: write tcp 188.166.225.170:33812->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:33812->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33812->145.79.227.4:4186: use of closed network connection Sep 27 19:19:54.974821864 (101h48m40.436914337s) [noise=true , address=145.79.227.4:4186 , piece_id=R4KDZI2P5HSD44V3YBFJVFGAWMMVPPKKVLNTKQSJTP7HZKABOHMA, seg_pos={0 0}, upload=89589]: Long tail cancellation due to optimal threshold Sep 27 19:19:54.974830510 (101h48m40.436922993s) [noise=true , address=45.91.116.10:4081 , piece_id=W4USXIFFW3ZU63ESEANFM5YGTLBRC4TZPFNNQZXTFCFMVLWEWHRA, seg_pos={0 0}, upload=89589]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF Sep 27 19:19:54.974839988 (101h48m40.436932461s) [noise=true , address=45.91.116.10:4081 , piece_id=W4USXIFFW3ZU63ESEANFM5YGTLBRC4TZPFNNQZXTFCFMVLWEWHRA, seg_pos={0 0}, upload=89589]: Long tail cancellation due to optimal threshold Sep 27 19:19:54.974882449 (101h48m40.436974922s) [seg_pos={0 0}, upload=89589]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 19:19:54.974976498 (101h48m40.437068971s) [upload=89589]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 19:19:54.974981578 (101h48m40.437074051s) [upload=89589]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 19:19:55.287167387 (101h48m40.74925986s) [upload=89589]: Done uploading segments... Sep 27 19:19:55.287193477 (101h48m40.749286191s) [upload=89589]: Upload finished. err: <nil> Duration: 17.627011939s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.368643489s FAIL (2025-09-27 18:50:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.368678966s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.094358283s FAIL (2025-09-27 18:49:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.094396877s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.632544603s FAIL (2025-09-27 18:48:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.632608765s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 9.59664177s FAIL (2025-09-27 18:48:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 9.596675405s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.354774867s FAIL (2025-09-27 18:47:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.354807879s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.110368469s FAIL (2025-09-27 18:46:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.110403076s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.64267323s FAIL (2025-09-27 18:45:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.642711583s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.930378623s FAIL (2025-09-27 18:44:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.930430241s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.259206786s FAIL (2025-09-27 18:43:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.259229942s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X........X..................................................] duration: 6.323582376s FAIL (2025-09-27 18:43:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.323630447s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X........X.................................................] duration: 5.91570847s FAIL (2025-09-27 18:42:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.915737966s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X........X................................................] duration: 6.042763604s FAIL (2025-09-27 18:41:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== entBegin Sep 27 18:06:12.378473535 (100h34m57.840566018s) [seg_pos={0 0}, upload=88496]: Begin upload segment... Sep 27 18:06:12.378546414 (100h34m57.840638897s) [seg_pos={0 0}, upload=88496]: Done begin upload segment. Sep 27 18:06:12.378576762 (100h34m57.840669245s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378605096 (100h34m57.840697579s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.378671422 (100h34m57.840763895s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379089538 (100h34m57.841182021s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379430236 (100h34m57.841522709s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.379741719 (100h34m57.841834203s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.380066479 (100h34m57.842158962s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.381196560 (100h34m57.843289033s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.382535941 (100h34m57.844628424s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.383819715 (100h34m57.845912199s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.385605836 (100h34m57.847698319s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.392225651 (100h34m57.854318134s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.394956130 (100h34m57.857048613s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.395534332 (100h34m57.857626815s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.396809290 (100h34m57.858901773s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:12.398237880 (100h34m57.860330363s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Uploading piece... Sep 27 18:06:14.026334352 (100h34m59.488426835s) [upload=88496]: Next returned nil segment Sep 27 18:06:14.026362976 (100h34m59.48845546s) [upload=88496]: Waiting for error group managing segments... Sep 27 18:06:14.410088039 (100h34m59.872180522s) [noise=true , address=145.79.227.12:4765 , piece_id=2DPSHW3GFR4JCS6AADLS7QSWCQJKL6UWHDI2ZBE6HEYVQ3RF62HQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.415625934 (100h34m59.877718417s) [noise=true , address=145.79.227.7:4294 , piece_id=AXGOLKP3XEYJ3BG52WM6RJFSWUUABBHJCBSTEAUSAHQHLBYASNNA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.443581767 (100h34m59.90567424s) [noise=true , address=145.79.227.14:4838 , piece_id=NFOWP2Z5TW4DE6DYMBYUJVSRS6D6HXGD6LMQR2EHBPOAHFCUMLXQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.444229030 (100h34m59.906321513s) [noise=true , address=145.79.227.18:4006 , piece_id=TUT4NZEITI6K4D3PQKEGMJVOMBIFHE3IKYFOWOSHUI5FER5P6D2A, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450151938 (100h34m59.912244421s) [noise=true , address=145.79.227.17:4970 , piece_id=IANY6PHDCP5XS2QRSALPA6SMJYAJZOBVHAGHXRKGEBWHY33UXUFQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.450594500 (100h34m59.912686973s) [noise=true , address=145.79.227.4:4187 , piece_id=XFDQTOZBIYLDTQXFJD2HY2XKFEJ2IYKKY3KOTLUVWXJII6O7QCBA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.599269735 (100h35m0.061362238s) [noise=true , address=45.91.116.12:4226 , piece_id=U6FCUO3FOMKYPIKTHIKW2PPD62UTSOFPKUQE547BBDRJ3AMX7AYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.778697087 (100h35m0.24078957s) [noise=true , address=45.91.116.7:4009 , piece_id=OCT7GC7JXTS6YSMSYTJUCVU2V3VE7EYQKOYCC4N7FRX34XDQLGPQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.879989948 (100h35m0.342082421s) [noise=true , address=45.91.116.13:4153 , piece_id=HAJ4XU6FBV4ZOV35AKCZ6RKW55NY5HUF447RAMFKJ6UMA6SEQ7BA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:14.900190743 (100h35m0.362283226s) [noise=true , address=45.91.116.5:4298 , piece_id=IUCMRDAU2WKVVBZDC7G2PKQAKMKJBQO76EPIHFH6SXA6M3RDHUKQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: <nil> Sep 27 18:06:41.319388066 (100h35m26.781480549s) [noise=true , address=77.90.31.19:4842 , piece_id=POIIIIE2EYNQWFYWQHR45RP3SUBRSXSOIIIHG6W55I4QGJKTC4TQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60194->77.90.31.19:4842: use of closed network connection Sep 27 18:06:41.319477597 (100h35m26.78157008s) [noise=true , address=77.90.31.19:4843 , piece_id=5IFYGX7KMTH5ZDI7CD5JCOOYLFEVEHXBP6VAFFNIU4H46KJY4ZEA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35810->77.90.31.19:4843: use of closed network connection Sep 27 18:06:41.319493146 (100h35m26.781585619s) [noise=true , address=77.90.31.13:4627 , piece_id=WUN5FGMYAESCZFEARR7VBA3JSG6WSYRPSP4VRGQAVDD4QAPPUMYQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60166->77.90.31.13:4627: use of closed network connection Sep 27 18:06:41.319547580 (100h35m26.781640063s) [noise=true , address=77.90.31.17:4770 , piece_id=H2PFBVPZG3FLAPVEGEAUK7VRTDZXG3O22JNEJAJEDGHAOCC46LXA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58004->77.90.31.17:4770: use of closed network connection Sep 27 18:06:41.319622282 (100h35m26.781714795s) [noise=true , address=77.90.31.11:4555 , piece_id=ZAXWGB3WO7JQNN7ESQ23ECXY5PSQARHKDVJ6EZM3RH6R3WMWMEKA, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->77.90.31.11:4555: use of closed network connection Sep 27 18:06:41.319717053 (100h35m26.781809536s) [noise=true , address=77.90.31.18:4807 , piece_id=HLDKPEWILT37NF2LEGB7T3T2PV2ZN4TMOPWNSQIGBH5F63YQ5OTQ, seg_pos={0 0}, upload=88496]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49752->77.90.31.18:4807: use of closed network connection Sep 27 18:06:41.319771646 (100h35m26.78186412s) [seg_pos={0 0}, upload=88496]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 18:06:41.600654288 (100h35m27.062746771s) [upload=88496]: Done uploading segments... Sep 27 18:06:41.600696929 (100h35m27.062789412s) [upload=88496]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.042794362s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.935843537s FAIL (2025-09-27 18:41:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: eestream: quiescence ======LOGS====== HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045225103 (100h27m26.507317576s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045696390 (100h27m26.507788863s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.045754741 (100h27m26.507847224s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046248521 (100h27m26.508341005s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046377137 (100h27m26.50846961s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046751550 (100h27m26.508844023s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.046881076 (100h27m26.508973559s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047117196 (100h27m26.509209679s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.047544259 (100h27m26.509636742s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048044812 (100h27m26.510137285s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048519366 (100h27m26.510611839s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.048832221 (100h27m26.510924704s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049376658 (100h27m26.511469141s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.049951122 (100h27m26.512043606s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:41.050335605 (100h27m26.512428088s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Uploading piece... Sep 27 17:58:42.529632646 (100h27m27.991725129s) [upload=88383]: Next returned nil segment Sep 27 17:58:42.529657103 (100h27m27.991749586s) [upload=88383]: Waiting for error group managing segments... Sep 27 17:58:42.961559761 (100h27m28.423652244s) [noise=true , address=145.79.227.41:4105 , piece_id=L4EX2IA6SYSEDDO5QZY24S5CMQBMTWF3SSLBPN4YC3TGZEJPDTGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.966984421 (100h27m28.429076894s) [noise=true , address=145.79.227.47:4287 , piece_id=TBCSLAFSHWDBII6GHVYATKUS2LKGGJ23UZTTWEX7B2NAJIFS3ZHQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.971845347 (100h27m28.43393782s) [noise=true , address=145.79.227.52:4458 , piece_id=PL76GT5HR577DOOBXQZOTXZ5UCOZVGE5UN2CXYHJMLUL5MQQB2RQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.978435966 (100h27m28.440528449s) [noise=true , address=145.79.227.21:4506 , piece_id=ZU67FHFUPBH556KIKS7R43YA4J6H53DAM7QKNOJB3MWXZIXORZGA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.981942373 (100h27m28.444034857s) [noise=true , address=145.79.227.42:4139 , piece_id=IJZX4SXVC35MTHGHUJSHFNVQ3N2BQDCB7QQ3OHSSKE64QMJXVXVQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:42.982097198 (100h27m28.444189681s) [noise=true , address=145.79.227.20:4451 , piece_id=Z5LDWRUTZNNOUAIJ3PKC64HBZOUOVE73TAPZF3ACKJFN2CB6Q3UQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.007382616 (100h27m28.469475099s) [noise=true , address=145.79.227.38:4994 , piece_id=YR42OLZOIRX2D4JDLQFXNPAHWJEMWEIRMME3EE3AZE3NSRZS7HDA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.248234892 (100h27m28.710327375s) [noise=true , address=45.91.116.5:4328 , piece_id=AKGWHKIKPDNVFAAQBK35LWM6SXKHQRXV2FD4KRQWRLGV3DXEB3AA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.427137994 (100h27m28.889230477s) [noise=true , address=45.91.116.7:4013 , piece_id=NN7JBELUCA4VMVUZM55GRFZSGEYCUDXDWNRNMWHR6KPZJMWN3XOQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:43.587565959 (100h27m29.049658432s) [noise=true , address=45.91.116.6:4998 , piece_id=4HOM77BLCDSISRVU7IMSKAMAN2BA5LLWFQZUZ2TTVPPNVT64YHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.678594859 (100h27m35.140687342s) [noise=true , address=77.90.31.18:4825 , piece_id=26HDVLWFREHZRUXMRF32MIVZJE5OPGIW2RYPYW33M5O7NMLAMVFA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.788334733 (100h27m35.250427206s) [noise=true , address=77.90.31.15:4730 , piece_id=JWLM663RZT4V5XNKURPZHNYKSKNWAD2ZSQ3ULYII5V7OTFESXZKA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.819080428 (100h27m35.281172921s) [noise=true , address=77.90.31.10:4538 , piece_id=3SFY3XR3NHI4U23GV2QM732FPL7A2DOKDOFGWCWMZQ7CJJ7SOFZA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851449795 (100h27m35.313542268s) [noise=true , address=77.90.31.14:4666 , piece_id=XO26QFGLFVDRZVXIIOQEG6YGJPTONQM5GJHXXRQ47UPKVBZSIHNA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: <nil> Sep 27 17:58:49.851477377 (100h27m35.313569861s) [seg_pos={0 0}, upload=88383]: Segment reached optimal threshold of 14 pieces. Sep 27 17:58:49.851615591 (100h27m35.313708074s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:191MKpEnoarS7sK4Gs63ZM8zGc5jevFX8VeqViP82duW33nJJz): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851624147 (100h27m35.31371663s) [noise=true , address=77.90.31.21:4883 , piece_id=FGZOB436YNE2UD4KBPZET6MVBCI57YTZYHNO4H2VDH3XZKDZBNQA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851753283 (100h27m35.313845766s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17ddUMj4ZejcjjWGzkGEM63vbCoEPnu7ePznzAPyzQofW8Q4gd): protocol: expected piece hash; context canceled; EOF Sep 27 17:58:49.851761078 (100h27m35.313853551s) [noise=true , address=77.90.31.19:4860 , piece_id=KDPQOHDXZJ4JTURTR5D4J4O5KSHY3YQLERUCIYSUX3EDRI55ZAWQ, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.851983270 (100h27m35.314075753s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xdfVQUEMLaGgw7hsBtHZVTKL9BTTxPeYve53efw8GPdSQDL9D): protocol: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49722->77.90.31.13:4642: use of closed network connection Sep 27 17:58:49.851993030 (100h27m35.314085503s) [noise=true , address=77.90.31.13:4642 , piece_id=YBHBLJIE4V7BUIYQZUAUA55CPJMHXJ2JV27M6D3PQRXQXW7OOFMA, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852275547 (100h27m35.31436804s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Qi796rSqngMJNBiFji2g4eYRn5MsqCqhpGPdYPN2GN8yFGTUx): protocol: read tcp 188.166.225.170:44604->77.90.31.21:4898: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:58:49.852285286 (100h27m35.314377769s) [noise=true , address=77.90.31.21:4898 , piece_id=KN2K5KCUIWWPX5IFHX45RD5OKBDAROT47VCXRD5HBJMVZZALYS5A, seg_pos={0 0}, upload=88383]: Long tail cancellation due to optimal threshold Sep 27 17:58:49.852302960 (100h27m35.314395443s) [seg_pos={0 0}, upload=88383]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Sep 27 17:58:49.852385116 (100h27m35.314477599s) [upload=88383]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 17:58:49.852391608 (100h27m35.314484091s) [upload=88383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 17:58:50.178450952 (100h27m35.640543435s) [upload=88383]: Done uploading segments... Sep 27 17:58:50.178476801 (100h27m35.640569284s) [upload=88383]: Upload finished. err: <nil> Duration: 5.935876751s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)