Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.25284488s FAIL (2025-07-16 08:00:30 UTC) orbiter warning: read tcp 188.166.225.170:39790->45.38.24.2:443: read: connection reset by peer Duration: 6.252874597s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.810413899s FAIL (2025-07-16 07:59:51 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.810437335s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.971076062s FAIL (2025-07-16 07:59:18 UTC) orbiter warning: read tcp 188.166.225.170:39790->45.38.24.2:443: read: connection reset by peer Duration: 5.971094739s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.830838684s FAIL (2025-07-16 07:59:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.830858213s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 6.517279875s FAIL (2025-07-16 07:58:55 UTC) orbiter warning: read tcp 188.166.225.170:39790->45.38.24.2:443: read: connection reset by peer Duration: 6.517295395s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.782679805s FAIL (2025-07-16 07:58:43 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.782709602s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.902768282s FAIL (2025-07-16 07:58:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.902810804s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.907971137s FAIL (2025-07-16 07:57:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.907996746s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.821942634s FAIL (2025-07-16 07:56:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.821974064s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.937884491s FAIL (2025-07-16 07:56:06 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.937908337s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.803814009s FAIL (2025-07-16 07:55:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.80384106s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.88660072s FAIL (2025-07-16 07:54:48 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.886622321s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.052354914s FAIL (2025-07-16 07:53:46 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.052384291s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.105030381s FAIL (2025-07-16 07:53:02 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.105057633s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 7.948395624s FAIL (2025-07-16 07:52:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 7.948444317s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.88101047s FAIL (2025-07-16 07:51:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.88103705s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 5.867843994s FAIL (2025-07-16 07:50:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.867873049s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.871467688s FAIL (2025-07-16 07:49:43 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.871498768s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 5.823382466s FAIL (2025-07-16 07:49:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.823400731s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 6.484040343s FAIL (2025-07-16 07:48:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.48407568s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.804380169s FAIL (2025-07-16 07:48:03 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.804414875s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.812769969s FAIL (2025-07-16 07:46:51 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.812884066s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.797098314s FAIL (2025-07-16 07:46:33 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.797128833s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.860780244s FAIL (2025-07-16 07:45:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.86080426s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.756068776s FAIL (2025-07-16 07:44:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.756096629s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.902290669s FAIL (2025-07-16 07:43:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.902322609s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.915461793s FAIL (2025-07-16 07:43:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.915494486s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.886311838s FAIL (2025-07-16 07:42:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.88633841s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 5.863240775s FAIL (2025-07-16 07:41:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.863267888s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.907736463s FAIL (2025-07-16 07:40:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.907760609s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.788647405s FAIL (2025-07-16 07:39:48 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.78867525s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.897602049s FAIL (2025-07-16 07:39:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.89762882s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.784556879s FAIL (2025-07-16 07:38:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.784586176s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.909878329s FAIL (2025-07-16 07:37:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.909897215s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.7617156s FAIL (2025-07-16 07:36:37 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.761743173s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 7.82849659s FAIL (2025-07-16 07:36:13 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 7.828516508s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.865883632s FAIL (2025-07-16 07:35:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.865908069s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.84779188s FAIL (2025-07-16 07:35:01 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.847819464s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.859642968s FAIL (2025-07-16 07:34:43 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.859673196s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.855460881s FAIL (2025-07-16 07:34:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.85552844s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.79948517s FAIL (2025-07-16 07:33:15 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.799506611s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.834898147s FAIL (2025-07-16 07:32:42 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.834921171s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.776168023s FAIL (2025-07-16 07:31:36 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.776217268s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.945510329s FAIL (2025-07-16 07:30:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.945542231s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 16.927080058s FAIL (2025-07-16 07:30:18 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 16.927114365s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X..........................................X................] duration: 5.711864501s FAIL (2025-07-16 07:29:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.711884449s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X..........................................X...............] duration: 5.870448302s FAIL (2025-07-16 07:29:00 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.870474992s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X..........................................X..............] duration: 5.830088024s FAIL (2025-07-16 07:28:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.830113252s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X..........................................X.............] duration: 5.861580898s FAIL (2025-07-16 07:27:43 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.861609684s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X..........................................X............] duration: 5.911161745s FAIL (2025-07-16 07:27:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 16 07:15:30.670329602 (550h5m5.440970937s) [upload=439963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 07:15:30.670333840 (550h5m5.440975175s) [upload=439963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 07:15:31.037176180 (550h5m5.807817515s) [upload=439963]: Done uploading segments... Jul 16 07:15:31.037192481 (550h5m5.807833816s) [upload=439963]: Upload finished. err: <nil> Jul 16 07:16:26.141279228 (550h6m0.911920563s) [upload=439972]: Starting upload Jul 16 07:16:26.141337289 (550h6m0.911978624s) [upload=439972]: Done starting upload Jul 16 07:16:26.141414086 (550h6m0.912055421s) [upload=439972]: Uploading segments... Jul 16 07:16:26.141498438 (550h6m0.912139773s) [upload=439972]: Got next segment. Inline: false Jul 16 07:16:26.141527563 (550h6m0.912168898s) [upload=439972]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 07:16:26.141532352 (550h6m0.912173687s) [upload=439972]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 07:16:26.387810013 (550h6m1.158451349s) [seg_pos={0 0}, upload=439972]: Begin upload segment... Jul 16 07:16:26.387861622 (550h6m1.158502957s) [seg_pos={0 0}, upload=439972]: Done begin upload segment. Jul 16 07:16:26.387881170 (550h6m1.158522505s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.387891469 (550h6m1.158532804s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388418347 (550h6m1.159059692s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388553045 (550h6m1.15919438s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.388738039 (550h6m1.159379374s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389036210 (550h6m1.159677555s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389074192 (550h6m1.159715537s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389356773 (550h6m1.159998108s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389666505 (550h6m1.16030784s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.389995464 (550h6m1.160636799s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390315076 (550h6m1.160956411s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390629748 (550h6m1.161271083s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.390961443 (550h6m1.161602779s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391269994 (550h6m1.161911329s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391287136 (550h6m1.161928471s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:26.391562964 (550h6m1.162204299s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Uploading piece... Jul 16 07:16:28.097522755 (550h6m2.86816408s) [upload=439972]: Next returned nil segment Jul 16 07:16:28.097536410 (550h6m2.868177745s) [upload=439972]: Waiting for error group managing segments... Jul 16 07:16:28.443256546 (550h6m3.213897881s) [noise=true , address=45.91.116.9:4262 , piece_id=6SIQTLHAZIFNCLKAIGYDT5E2MEJOL5AYU53KHJ3E7CFUII7TOFLQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.443612908 (550h6m3.214254243s) [noise=true , address=104.36.82.15:4257 , piece_id=GSO5VDQPEFQEQS3K5RE2IFJDD44QNEGYT37MABJRL6IUR4QKRSUA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.452354063 (550h6m3.222995398s) [noise=true , address=45.91.116.7:4011 , piece_id=PAY5TW7MTQMCEJGJQLEZHOIPALLTGL32XWSIHKWAPHCKGC4HZWWA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.455240004 (550h6m3.22588133s) [noise=true , address=104.36.82.4:4802 , piece_id=3QFJ226OZEPUS4M4SOEH5WR34UQAQJUEN5DYXL6CTNCHC23PFMEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.516897034 (550h6m3.28753837s) [noise=true , address=104.36.82.15:4258 , piece_id=3P4DX3R4NUT3CHWPCTZM6JQJSVUVV7WCNQC3R3OTSIFNWW2YQFHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.520076909 (550h6m3.290718244s) [noise=true , address=45.91.116.10:4082 , piece_id=EKGKMBPPUW546X56QOQBJKFUF3ZEDCC7FBNXVIGK6H4ASQNBGOCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.526746490 (550h6m3.297387825s) [noise=true , address=104.36.82.7:4911 , piece_id=PXX5CVBX2STHL7ADJCSURJ3RMP2XTE67AAXNUNFUDLQT3DFBEW5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.531731768 (550h6m3.302373103s) [noise=true , address=45.91.116.5:4297 , piece_id=EPARBDZL267E7JMG6Z3O7O6LKUUYQ5UER2YJ2EY7R4R2FDHVAJ5A, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.535121835 (550h6m3.30576317s) [noise=true , address=104.36.82.9:4006 , piece_id=TSIRZ73376HNFDZTOGLGA3DY4YYRBGIEMKN26X2MVZVXTVW4QKMA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:28.537995282 (550h6m3.308636618s) [noise=true , address=45.91.116.13:4153 , piece_id=EUFVSQ3NZYYYOSXA4TFDVZWIPBREJB3S7SQMAOJQJ2QE6WX3CIAQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:47.015597962 (550h6m21.786239297s) [noise=true , address=77.90.31.11:4553 , piece_id=NIF4ZAQ5EFO43T7ARDL6HXSCY6EFBG7RMHKFNCVCMXU5PDF3MPBQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:53.793163570 (550h6m28.563804905s) [noise=true , address=77.90.31.11:4554 , piece_id=LHSN7KOLRT22Q7NNEB5RQNEFDQ4IIY3QUTXX5HCAM7IBOGS46FEQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: <nil> Jul 16 07:16:55.462521041 (550h6m30.233162376s) [noise=true , address=77.90.31.12:4592 , piece_id=3DMXCH3QSPBBMOLFI2EHKBKJPVMGPPJ77IRNZK6O4G27FKUAY6NA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 07:16:55.462563902 (550h6m30.233205237s) [noise=true , address=77.90.31.15:4698 , piece_id=PN36J7QXWABEWU2EXU2G5GDF4CMRUHF4SWZOKAVYOFXL54HD66VQ, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42394->77.90.31.15:4698: use of closed network connection Jul 16 07:16:55.462588419 (550h6m30.233229754s) [noise=true , address=77.90.31.19:4843 , piece_id=Q6UY6TXDDRPPO67O4KSRYLJ4DYYHDGLIY7KTYHQS65HHQWKYLQHA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->77.90.31.19:4843: use of closed network connection Jul 16 07:16:55.462604440 (550h6m30.233245765s) [noise=true , address=77.90.31.10:4518 , piece_id=KQC3JEKH4CFPE3KXTV4LADBDLIJ6IIROOVPA3Y6CFVM6ZBVUAXCA, seg_pos={0 0}, upload=439972]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55992->77.90.31.10:4518: use of closed network connection Jul 16 07:16:55.462626863 (550h6m30.233268198s) [seg_pos={0 0}, upload=439972]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 07:16:55.702313482 (550h6m30.472954817s) [upload=439972]: Done uploading segments... Jul 16 07:16:55.702339622 (550h6m30.472980957s) [upload=439972]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.911191271s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)