Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.70303964s FAIL (2025-07-05 15:33:46 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.703069366s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.664789474s FAIL (2025-07-05 15:32:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.664822246s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.773752479s FAIL (2025-07-05 15:32:00 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.773781204s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.671639495s FAIL (2025-07-05 15:31:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.671680052s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.786413682s FAIL (2025-07-05 15:30:16 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.786436666s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.656734009s FAIL (2025-07-05 15:29:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.656758065s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 5.606961592s FAIL (2025-07-05 15:28:31 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.606980578s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.685936169s FAIL (2025-07-05 15:28:14 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.6859629s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.654957151s FAIL (2025-07-05 15:27:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.654982802s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.706703709s FAIL (2025-07-05 15:26:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.706727555s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.720192924s FAIL (2025-07-05 15:25:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.720217582s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 5.745733783s FAIL (2025-07-05 15:25:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.745758449s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 5.843993466s FAIL (2025-07-05 15:24:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.844022501s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 5.650977849s FAIL (2025-07-05 15:23:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.651002715s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 5.65965438s FAIL (2025-07-05 15:23:18 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.659678355s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.752026351s FAIL (2025-07-05 15:22:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.752059814s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.655314348s FAIL (2025-07-05 15:22:14 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.655340969s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.835638043s FAIL (2025-07-05 15:21:08 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.835665825s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.861067631s FAIL (2025-07-05 15:20:31 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.861095855s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.786134617s FAIL (2025-07-05 15:19:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.78616271s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.605427097s FAIL (2025-07-05 15:18:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.605445822s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.242854304s FAIL (2025-07-05 15:18:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== JYE6FC5T5QIBUQIILGDHNGXAUDBUIAOSKYFMPBCWPHP2SDJ3A, seg_pos={0 0}, upload=235214]: Done uploading piece. err: <nil> Jul 5 15:17:12.446386719 (294h6m47.217028064s) [seg_pos={0 0}, upload=235214]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 5 15:17:12.446476771 (294h6m47.217118106s) [upload=235214]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 5 15:17:12.446483043 (294h6m47.217124378s) [upload=235214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 5 15:17:12.905279085 (294h6m47.67592042s) [upload=235214]: Done uploading segments... Jul 5 15:17:12.905303211 (294h6m47.675944546s) [upload=235214]: Upload finished. err: <nil> Jul 5 15:18:10.381454266 (294h7m45.152095611s) [upload=235227]: Starting upload Jul 5 15:18:10.381524069 (294h7m45.152165414s) [upload=235227]: Done starting upload Jul 5 15:18:10.381621115 (294h7m45.15226245s) [upload=235227]: Uploading segments... Jul 5 15:18:10.381740133 (294h7m45.152381468s) [upload=235227]: Got next segment. Inline: false Jul 5 15:18:10.381776342 (294h7m45.152417677s) [upload=235227]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 5 15:18:10.381781973 (294h7m45.152423308s) [upload=235227]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 5 15:18:10.613904835 (294h7m45.38454617s) [seg_pos={0 0}, upload=235227]: Begin upload segment... Jul 5 15:18:10.613947397 (294h7m45.384588732s) [seg_pos={0 0}, upload=235227]: Done begin upload segment. Jul 5 15:18:10.613968548 (294h7m45.384609883s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.613995038 (294h7m45.384636373s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614303969 (294h7m45.384945304s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614367721 (294h7m45.385009056s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614619272 (294h7m45.385260607s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614637758 (294h7m45.385279083s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.614890251 (294h7m45.385531576s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615197719 (294h7m45.385839054s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615413552 (294h7m45.386054887s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615608124 (294h7m45.386249459s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615803147 (294h7m45.386444472s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.615998511 (294h7m45.386639836s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616193414 (294h7m45.386834749s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616449093 (294h7m45.387090429s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616635840 (294h7m45.387277175s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:10.616818790 (294h7m45.387460125s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Uploading piece... Jul 5 15:18:12.265123988 (294h7m47.035765313s) [upload=235227]: Next returned nil segment Jul 5 15:18:12.265138164 (294h7m47.035779499s) [upload=235227]: Waiting for error group managing segments... Jul 5 15:18:12.550465694 (294h7m47.321107019s) [noise=true , address=104.36.82.8:4970 , piece_id=2YJN7VKTTTCMAWQW2MNR65DAJPFMHNIBIW5NWYNG4R6HNPRFH6NQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.554315629 (294h7m47.324956954s) [noise=true , address=104.36.82.12:4149 , piece_id=YBTLQEWDMFK6JIFTYR3VJDNMXTS3NMWWHX2MPP53BM2UBDI3UUEA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.603452512 (294h7m47.374093847s) [noise=true , address=104.36.82.3:4767 , piece_id=OGHG4G5ZVC7GVY4XI6DO4KINXF7THHL6IHGDB6DAPDNUDOUFSC4A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639796643 (294h7m47.410437978s) [noise=true , address=104.36.82.9:4006 , piece_id=47FME2B5NS6WMOHC2ZGE2FZVSPEQUXKYT6E7AOSANAE5XZVDH5VQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.639894960 (294h7m47.410536285s) [noise=true , address=104.36.82.15:4259 , piece_id=LQKQ5MGKXHTPITBR2QAOQ5CX76GEVJGHJT3ZD3G4PFNN5IBTCJBQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.647222659 (294h7m47.417863994s) [noise=true , address=104.36.82.16:4295 , piece_id=VEL6CAUOJKMRN7A7M66ZPDLCMJVD555LZPHM53A24MDQJQOW7ECQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.665424010 (294h7m47.436065346s) [noise=true , address=104.36.82.13:4185 , piece_id=LTQS2CUZ5NRJCBOPEXDKUQ2IJGOIW3JJAPHWTCW7WUMQZPIWRXRA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.702209574 (294h7m47.47285091s) [noise=true , address=45.91.116.14:4191 , piece_id=WH2BCI36GGFF4QUD4XDYBBDMUR2R2FFVYKFTZD3ZWXPJCFJ3D7HQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.722073145 (294h7m47.49271448s) [noise=true , address=45.91.116.5:4298 , piece_id=TVTUGWHFTXB32GBY4LYXT3YDB5LIFXEGA7VO5BHFA24Y6EMDTPHQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:12.788621529 (294h7m47.559262864s) [noise=true , address=45.91.116.4:4937 , piece_id=YMKIHJET2EN3QHEK665ACZNQNHJ3UGE4H5T4DIHWJUHJQBYO7B7Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:36.752254721 (294h8m11.522896046s) [noise=true , address=77.90.31.19:4842 , piece_id=R4FTRBZ3PETCA6QGU6RPUEDZZBWXF7YVH2COEW4WH3YWDLU3VJMA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:37.783437003 (294h8m12.554078338s) [noise=true , address=77.90.31.17:4771 , piece_id=V7M65ZHCGXPD2GVJL6LTB4NP3QN6QLSIBM7U34QAYPAIKMN6EBPA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: <nil> Jul 5 15:18:39.733398290 (294h8m14.504039626s) [noise=true , address=77.90.31.18:4805 , piece_id=I23F6KYUQ7TJI4ZLEANACVNYJDMPKKMMACIZAIA6PC5S3CKQPV2Q, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733464387 (294h8m14.504105722s) [noise=true , address=77.90.31.21:4877 , piece_id=OJBIFN2TJPQ45LQLYCMPKV5UZM6Y2MXNOBT3TSGX26U4SZAL5Q7A, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: expected piece hash; context deadline exceeded; EOF Jul 5 15:18:39.733542065 (294h8m14.50418339s) [noise=true , address=77.90.31.11:4554 , piece_id=4QRZ5MPQ7HTPRIVRL3O3H4OV7CQA7ESYT5RCPI3GTHXJOHOSAOVA, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38250->77.90.31.11:4554: use of closed network connection Jul 5 15:18:39.733547496 (294h8m14.504188831s) [noise=true , address=77.90.31.16:4733 , piece_id=3GQ7NEWE6WZJMQUA3GG3GSZYXEFNOAUEVWHOEH4TSNONYXUT43GQ, seg_pos={0 0}, upload=235227]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40656->77.90.31.16:4733: use of closed network connection Jul 5 15:18:39.733583284 (294h8m14.504224619s) [seg_pos={0 0}, upload=235227]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 5 15:18:39.974321220 (294h8m14.744962555s) [upload=235227]: Done uploading segments... Jul 5 15:18:39.974353041 (294h8m14.744994376s) [upload=235227]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.242874914s core uplink us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 759.294384ms FAIL (2025-07-05 14:10:19 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 759.325063ms edge authservice http us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 766.238449ms FAIL (2025-07-05 14:09:05 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 766.262796ms edge authservice http us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 759.738494ms FAIL (2025-07-05 14:08:13 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 759.760156ms edge authservice http us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 790.756395ms FAIL (2025-07-05 14:07:46 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 790.779519ms edge authservice http us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.52160683s FAIL (2025-07-05 14:06:46 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.521626529s edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 718.829592ms FAIL (2025-07-05 14:06:40 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 718.86ms edge authservice http us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.446165822s FAIL (2025-07-05 14:06:31 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.446204266s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.35743263s FAIL (2025-07-05 14:05:59 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.357451917s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.367650828s FAIL (2025-07-05 14:05:44 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.367674293s edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 788.736442ms FAIL (2025-07-05 14:05:39 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 788.78773ms edge authservice http us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 793.134944ms FAIL (2025-07-05 14:05:06 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 793.162898ms edge authservice http us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 2.239751539s FAIL (2025-07-05 14:04:37 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.239780114s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 11.532921945s FAIL (2025-07-05 14:04:11 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 11.532944458s edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 768.82043ms FAIL (2025-07-05 14:03:52 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 768.864434ms edge authservice http us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 2 out of 60 is failed (threshold=3) [X.......X...................................................] duration: 1.015819341s FAIL (2025-07-05 14:03:34 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 1.015851433s edge authservice http us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.406788403s FAIL (2025-07-05 14:03:01 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.40681342s edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 2 out of 60 is failed (threshold=3) [.X.......X..................................................] duration: 760.08182ms FAIL (2025-07-05 14:02:40 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 760.103562ms edge authservice http us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.366952832s FAIL (2025-07-05 14:02:20 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.366978841s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [X.......X...................................................] duration: 2.321380223s FAIL (2025-07-05 14:01:43 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.321399701s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.927769298s FAIL (2025-07-05 14:01:32 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 5.92779138s edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 2 out of 60 is failed (threshold=3) [..X.......X.................................................] duration: 750.636048ms FAIL (2025-07-05 14:01:29 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 750.666306ms edge authservice http us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.179169781s FAIL (2025-07-05 14:01:00 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 6.179245616s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.X.......X..................................................] duration: 2.353776943s FAIL (2025-07-05 14:00:59 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.353801288s edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 2 out of 60 is failed (threshold=3) [...X.......X................................................] duration: 765.563043ms FAIL (2025-07-05 14:00:56 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 765.591327ms edge authservice http us-select-1
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 501.186887ms target: http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt FAIL (2025-07-05 14:00:50 UTC) orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout Duration: 501.233086ms edge linksharing us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.553907478s FAIL (2025-07-05 14:00:15 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 6.553950319s edge gateway us-select-1
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 516.981475ms target: http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt FAIL (2025-07-05 14:00:14 UTC) orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout Duration: 516.999499ms edge linksharing us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 2 out of 60 is failed (threshold=3) [....X.......X...............................................] duration: 740.977129ms FAIL (2025-07-05 14:00:13 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 741.027104ms edge authservice http us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)