Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 10 out of 60 is failed (threshold=3) [.X..........XXXX.X.X.......X...X.................X..........] duration: 5.946491711s FAIL (2025-08-17 04:58:03 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 04:51:09.056938814 (203h25m3.488018935s) [upload=211594]: Starting upload Aug 17 04:51:09.057017635 (203h25m3.488097766s) [upload=211594]: Done starting upload Aug 17 04:51:09.057190195 (203h25m3.488270326s) [upload=211594]: Uploading segments... Aug 17 04:51:09.057308371 (203h25m3.488388502s) [upload=211594]: Got next segment. Inline: false Aug 17 04:51:09.057342866 (203h25m3.488422997s) [upload=211594]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:51:09.057347857 (203h25m3.488427978s) [upload=211594]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:51:09.504198039 (203h25m3.93527816s) [seg_pos={0 0}, upload=211594]: Begin upload segment... Aug 17 04:51:09.504246441 (203h25m3.935326572s) [seg_pos={0 0}, upload=211594]: Done begin upload segment. Aug 17 04:51:09.504264686 (203h25m3.935344817s) [noise=true , address=104.36.82.2:4748 , piece_id=T6MOSFTUKYGVT2YSCEGXMSDXLULB6NI4RGGGB3SSARO7ALNZJN5Q, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504308099 (203h25m3.93538823s) [noise=true , address=104.36.82.19:4436 , piece_id=VN6NIXXOT6KUQE3JU6X35TPYNCI5K4KOSBLVXMOKSP6N2SYRFM6Q, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504588896 (203h25m3.935669017s) [noise=true , address=104.36.82.58:4295 , piece_id=3LNVXIW5IU2BQZRH6QAVV7A2PJ3L3YSGNOXBMIYOJMFBVHJZUPYA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504650284 (203h25m3.935730405s) [noise=true , address=64.49.13.169:4706 , piece_id=Y4SS3PCBRVCQS5N5N34Y6PGUJ5FPWOCHHO5KMU2PJ5CUXWTHW7RQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504844445 (203h25m3.935924566s) [noise=true , address=45.91.116.12:4228 , piece_id=X5VGNN2WVGEW3E4MPUO5MQWZ7L2JMB345IQUUTBI4DYW5HESK33A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504957571 (203h25m3.936037702s) [noise=true , address=104.36.82.23:4475 , piece_id=TYHG2VUDQ4RRMFLUOLV3U73G7JXA6ZRC757TD34Q2MZTJNRMUIZA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505108992 (203h25m3.936189123s) [noise=true , address=104.36.82.59:4331 , piece_id=SPNJSZRY7LNIS7ZCHJLOOWKVZU7AP3CODFM73CGOLQZU7HUCT2IQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505263607 (203h25m3.936343738s) [noise=true , address=66.23.193.175:4553 , piece_id=BXVNQMDXCQPEUSLIZ24VI22F7FOOJN2MEGJU6UI2SDQIZJSFQJ2A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505341626 (203h25m3.936421757s) [noise=true , address=77.90.31.10:4523 , piece_id=22C56K4AOQCL5N6WR7QECVAAXHOS3OUNMGWCRFHP5GKSTHIT7OLA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505565765 (203h25m3.936645896s) [noise=true , address=104.36.82.4:4836 , piece_id=JJWCMFYVQUU6WGEVBEJS4JPZQNNZERI5BVTBBAQQUZV3WD63NGIA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505597326 (203h25m3.936677457s) [noise=true , address=77.90.31.11:4563 , piece_id=QDC72JPWTU4LH5URAU44ECPV72L4NQPWGDDOHHVYSGNJTFS7C6XQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505853546 (203h25m3.936933677s) [noise=true , address=104.36.82.18:4396 , piece_id=3CORBTMNXLB5AG6ZGAPLPBGGFEQ4SSMJHRGXPESWTJD5YPYVUKFA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505894053 (203h25m3.936974184s) [noise=true , address=104.36.82.44:4896 , piece_id=2DQKKZFJX4N6W5UOOYX4QWSZ3GAFBMQIIDCKU3N3N5MLM4Q7T2AQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506090339 (203h25m3.937170471s) [noise=true , address=77.90.31.13:4658 , piece_id=XONXAMV3VAHHSHNIDMAIMMD6KOCKLYJQAUTNVMSUXB6ZCWP5THDA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506165653 (203h25m3.937245784s) [noise=true , address=104.36.82.5:4850 , piece_id=6ZYI2JS5HMQKTUM2XR4ZTSJMC3MHVILV4YOQ7ANA5PX3FDJXJ2DA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506275653 (203h25m3.937355774s) [noise=true , address=45.91.116.14:4199 , piece_id=HOTL5QUTJLBBZNF6A4WPRKMQXB72I3TUVSDYUVWMU7G4ZIXOIK3A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506396365 (203h25m3.937476496s) [noise=true , address=104.36.82.34:4565 , piece_id=LUXKT7N4OUT4UJA345KTE2T3QUD3LEGLUQKQTDZADYBOGE2Z5XNQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506467330 (203h25m3.937547461s) [noise=true , address=104.36.82.60:4354 , piece_id=M3INHRZ2XHGLNFSI7OHKHKNCVTSOWTSCJJCGHTTYCAJGFLDWLL7A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:10.617749206 (203h25m5.048829337s) [upload=211594]: Next returned nil segment Aug 17 04:51:10.617765637 (203h25m5.048845758s) [upload=211594]: Waiting for error group managing segments... Aug 17 04:51:10.899081476 (203h25m5.330161607s) [noise=true , address=64.49.13.169:4706 , piece_id=Y4SS3PCBRVCQS5N5N34Y6PGUJ5FPWOCHHO5KMU2PJ5CUXWTHW7RQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:10.959773754 (203h25m5.390853875s) [noise=true , address=66.23.193.175:4553 , piece_id=BXVNQMDXCQPEUSLIZ24VI22F7FOOJN2MEGJU6UI2SDQIZJSFQJ2A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.387426163 (203h25m5.818506294s) [noise=true , address=77.90.31.11:4563 , piece_id=QDC72JPWTU4LH5URAU44ECPV72L4NQPWGDDOHHVYSGNJTFS7C6XQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.392712709 (203h25m5.82379284s) [noise=true , address=45.91.116.14:4199 , piece_id=HOTL5QUTJLBBZNF6A4WPRKMQXB72I3TUVSDYUVWMU7G4ZIXOIK3A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.399286516 (203h25m5.830366637s) [noise=true , address=104.36.82.4:4836 , piece_id=JJWCMFYVQUU6WGEVBEJS4JPZQNNZERI5BVTBBAQQUZV3WD63NGIA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.412750859 (203h25m5.84383099s) [noise=true , address=104.36.82.23:4475 , piece_id=TYHG2VUDQ4RRMFLUOLV3U73G7JXA6ZRC757TD34Q2MZTJNRMUIZA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.439625902 (203h25m5.870706033s) [noise=true , address=77.90.31.10:4523 , piece_id=22C56K4AOQCL5N6WR7QECVAAXHOS3OUNMGWCRFHP5GKSTHIT7OLA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.465577498 (203h25m5.896657629s) [noise=true , address=104.36.82.34:4565 , piece_id=LUXKT7N4OUT4UJA345KTE2T3QUD3LEGLUQKQTDZADYBOGE2Z5XNQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.469847087 (203h25m5.900927218s) [noise=true , address=104.36.82.60:4354 , piece_id=M3INHRZ2XHGLNFSI7OHKHKNCVTSOWTSCJJCGHTTYCAJGFLDWLL7A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.470697434 (203h25m5.901777555s) [noise=true , address=104.36.82.5:4850 , piece_id=6ZYI2JS5HMQKTUM2XR4ZTSJMC3MHVILV4YOQ7ANA5PX3FDJXJ2DA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.470949846 (203h25m5.902029978s) [noise=true , address=104.36.82.19:4436 , piece_id=VN6NIXXOT6KUQE3JU6X35TPYNCI5K4KOSBLVXMOKSP6N2SYRFM6Q, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.486560086 (203h25m5.917640217s) [noise=true , address=104.36.82.2:4748 , piece_id=T6MOSFTUKYGVT2YSCEGXMSDXLULB6NI4RGGGB3SSARO7ALNZJN5Q, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.498376395 (203h25m5.929456526s) [noise=true , address=104.36.82.44:4896 , piece_id=2DQKKZFJX4N6W5UOOYX4QWSZ3GAFBMQIIDCKU3N3N5MLM4Q7T2AQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.512939881 (203h25m5.944020012s) [noise=true , address=45.91.116.12:4228 , piece_id=X5VGNN2WVGEW3E4MPUO5MQWZ7L2JMB345IQUUTBI4DYW5HESK33A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.512969688 (203h25m5.944049819s) [seg_pos={0 0}, upload=211594]: Segment reached optimal threshold of 14 pieces. Aug 17 04:51:11.513156405 (203h25m5.944236536s) [noise=true , address=77.90.31.13:4658 , piece_id=XONXAMV3VAHHSHNIDMAIMMD6KOCKLYJQAUTNVMSUXB6ZCWP5THDA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129aongkQE9HMNz8vsNNA5SvKbhsHqdt9UrtSTPJJpP9mkFsboX): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513220778 (203h25m5.944300909s) [noise=true , address=104.36.82.58:4295 , piece_id=3LNVXIW5IU2BQZRH6QAVV7A2PJ3L3YSGNOXBMIYOJMFBVHJZUPYA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oLa34qwPBAx3CTvYYj7GN3oxxWGJ9dAYXfke1eUftZc6xzfqg): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513269471 (203h25m5.944349603s) [noise=true , address=104.36.82.59:4331 , piece_id=SPNJSZRY7LNIS7ZCHJLOOWKVZU7AP3CODFM73CGOLQZU7HUCT2IQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513316901 (203h25m5.944397033s) [noise=true , address=104.36.82.18:4396 , piece_id=3CORBTMNXLB5AG6ZGAPLPBGGFEQ4SSMJHRGXPESWTJD5YPYVUKFA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1k5gzJBGWhpARcfiynburgfTqJ2t7sVNs1mAkUfToMG5saxPqx): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513333754 (203h25m5.944413885s) [seg_pos={0 0}, upload=211594]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 04:51:11.513406654 (203h25m5.944486775s) [upload=211594]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 04:51:11.513424287 (203h25m5.944504418s) [upload=211594]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:51:11.809871930 (203h25m6.240952061s) [upload=211594]: Done uploading segments... Aug 17 04:51:11.809892119 (203h25m6.24097225s) [upload=211594]: Upload finished. err: <nil> Duration: 5.94652294s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.............................X.....X.....................XX.] duration: 1.984920752s FAIL (2025-08-17 04:58:02 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 04:56:50.046028598 (203h30m44.477108729s) [upload=211693]: Starting upload Aug 17 04:56:50.046078113 (203h30m44.477158244s) [upload=211693]: Done starting upload Aug 17 04:56:50.046127948 (203h30m44.477208079s) [upload=211693]: Uploading segments... Aug 17 04:56:50.046165199 (203h30m44.477245331s) [upload=211693]: Got next segment. Inline: true Aug 17 04:56:50.046169808 (203h30m44.477249929s) [upload=211693]: Waiting for error group managing segments... Aug 17 04:56:50.046201128 (203h30m44.477281249s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:56:50.046204875 (203h30m44.477285006s) [upload=211693]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:56:50.046207780 (203h30m44.477287911s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:56:50.545849463 (203h30m44.976929594s) [upload=211693]: Done uploading segments... Aug 17 04:56:50.545868880 (203h30m44.976949011s) [upload=211693]: Upload finished. err: <nil> Duration:* 1.984955018s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........XXXX...............................................] duration: 6.075354234s FAIL (2025-08-17 04:57:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 04:23:03.873697274 (202h56m58.304777405s) [upload=211112]: Starting upload Aug 17 04:23:03.873756368 (202h56m58.304836499s) [upload=211112]: Done starting upload Aug 17 04:23:03.873839848 (202h56m58.304919989s) [upload=211112]: Uploading segments... Aug 17 04:23:03.873950850 (202h56m58.305030981s) [upload=211112]: Got next segment. Inline: false Aug 17 04:23:03.873976509 (202h56m58.30505664s) [upload=211112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:03.873980757 (202h56m58.305060888s) [upload=211112]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:04.113727713 (202h56m58.544807844s) [seg_pos={0 0}, upload=211112]: Begin upload segment... Aug 17 04:23:04.113774051 (202h56m58.544854182s) [seg_pos={0 0}, upload=211112]: Done begin upload segment. Aug 17 04:23:04.113820890 (202h56m58.544901021s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.113845989 (202h56m58.54492612s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114195037 (202h56m58.545275168s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114281412 (202h56m58.545361543s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114482115 (202h56m58.545562237s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114570385 (202h56m58.545650516s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114757392 (202h56m58.545837523s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114874727 (202h56m58.545954858s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115030715 (202h56m58.546110846s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115194288 (202h56m58.546274419s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115307967 (202h56m58.546388098s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115535692 (202h56m58.546615823s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115754130 (202h56m58.546834261s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116079182 (202h56m58.547159323s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116350181 (202h56m58.547430312s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116554471 (202h56m58.547634602s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:05.868765063 (202h57m0.299845195s) [upload=211112]: Next returned nil segment Aug 17 04:23:05.868801043 (202h57m0.299881164s) [upload=211112]: Waiting for error group managing segments... Aug 17 04:23:06.147430529 (202h57m0.57851066s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.182400314 (202h57m0.613480445s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.191910532 (202h57m0.622990663s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.197391631 (202h57m0.628471782s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.198069487 (202h57m0.629149618s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.200597014 (202h57m0.631677145s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.288053251 (202h57m0.719133372s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.298810435 (202h57m0.729890566s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.301049520 (202h57m0.732129651s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.402776985 (202h57m0.833857116s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:33.223003412 (202h57m27.654083543s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: write tcp 188.166.225.170:42258->104.36.82.5:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223050702 (202h57m27.654130823s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:52308->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223090619 (202h57m27.65417075s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 188.166.225.170:53078->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223104976 (202h57m27.654185097s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:56176->104.36.82.4:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223149581 (202h57m27.654229712s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:40582->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223157536 (202h57m27.654237667s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 188.166.225.170:32954->104.36.82.9:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223177415 (202h57m27.654257536s) [seg_pos={0 0}, upload=211112]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 04:23:33.448862924 (202h57m27.879943045s) [upload=211112]: Done uploading segments... Aug 17 04:23:33.448886830 (202h57m27.879966961s) [upload=211112]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.075375435s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........XXXX...............................................] duration: 6.324508855s FAIL (2025-08-17 04:57:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.324560203s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...............XX...........................................] duration: 6.314724535s FAIL (2025-08-17 04:57:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.314745275s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........XXX...............................................] duration: 5.78479542s FAIL (2025-08-17 04:57:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== g_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761278730 (202h56m21.192358861s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761661883 (202h56m21.192742014s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761716548 (202h56m21.192796679s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761972016 (202h56m21.193052147s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762254878 (202h56m21.193335009s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762523682 (202h56m21.193603803s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762864294 (202h56m21.193944415s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763179537 (202h56m21.194259668s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763478499 (202h56m21.19455864s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763757683 (202h56m21.194837814s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763988195 (202h56m21.195068326s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764273320 (202h56m21.195353451s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764555100 (202h56m21.195635221s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764757416 (202h56m21.195837547s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.765868993 (202h56m21.196949124s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.767152809 (202h56m21.19823294s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.768577886 (202h56m21.199658017s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:27.934309011 (202h56m22.365389142s) [upload=211101]: Next returned nil segment Aug 17 04:22:27.934342545 (202h56m22.365422666s) [upload=211101]: Waiting for error group managing segments... Aug 17 04:22:28.231985083 (202h56m22.663065214s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.670701256 (202h56m23.101781427s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.671952431 (202h56m23.103032562s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.698468278 (202h56m23.129548409s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.720171187 (202h56m23.151251318s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.731591429 (202h56m23.16267156s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.798148986 (202h56m23.229229117s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.804849037 (202h56m23.235929168s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.833066822 (202h56m23.264147013s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.846684710 (202h56m23.277764831s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:23:25.582810947 (202h57m20.013891078s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12TU4zvu38mF1fNSergCtMj2cK9eu57Rdfpt5MVk3tc4MZF6dNr, address:104.36.82.34:4558): protocol: rpc: dial tcp 104.36.82.34:4558: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582856475 (202h57m20.013936606s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12D9nh4XFmsjjFcZEMjaL4uw4Whsa8nfMrAsYPu6NVpsxgb3jvf, address:104.36.82.35:4594): protocol: rpc: dial tcp 104.36.82.35:4594: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582973238 (202h57m20.014053369s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1WvyEw8XevCiQ8yRgzyMo7WTVzpEJ158ErKfAeV4dSTBkS1oRF, address:104.36.82.19:4436): protocol: rpc: dial tcp 104.36.82.19:4436: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583033393 (202h57m20.014113524s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:104.36.82.58:4283): protocol: rpc: dial tcp 104.36.82.58:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583069241 (202h57m20.014149362s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:126Z6tvp49dLLdj3T5snJc4tBVGzcX5UBaGafFN3PkhF92vxpfs, address:104.36.82.50:4047): protocol: rpc: dial tcp 104.36.82.50:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583076716 (202h57m20.014156837s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1zzW2z6noGQT6xdSmwSQBNYwip6gZjnmqZg7qdb6BP3nHyCLFz, address:104.36.82.39:4753): protocol: rpc: dial tcp 104.36.82.39:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583107535 (202h57m20.014187666s) [noise=true , address=104.36.82.23:4474 , piece_id=RITPHITVW655UQETT6UKS7BZO2S3F4P2MJIO4REOUKYYP4E2CDVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 188.166.225.170:36642->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583165355 (202h57m20.014245487s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh, address:104.36.82.5:4855): protocol: write tcp 188.166.225.170:47052->104.36.82.5:4855: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583194000 (202h57m20.014274131s) [seg_pos={0 0}, upload=211101]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:25.810738626 (202h57m20.241818757s) [upload=211101]: Done uploading segments... Aug 17 04:23:25.810761991 (202h57m20.241842122s) [upload=211101]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.784827631s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 10 out of 60 is failed (threshold=3) [..X..........XXXX.X.X.......X...X.................X.........] duration: 5.802183629s FAIL (2025-08-17 04:56:52 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 04:51:09.056938814 (203h25m3.488018935s) [upload=211594]: Starting upload Aug 17 04:51:09.057017635 (203h25m3.488097766s) [upload=211594]: Done starting upload Aug 17 04:51:09.057190195 (203h25m3.488270326s) [upload=211594]: Uploading segments... Aug 17 04:51:09.057308371 (203h25m3.488388502s) [upload=211594]: Got next segment. Inline: false Aug 17 04:51:09.057342866 (203h25m3.488422997s) [upload=211594]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:51:09.057347857 (203h25m3.488427978s) [upload=211594]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:51:09.504198039 (203h25m3.93527816s) [seg_pos={0 0}, upload=211594]: Begin upload segment... Aug 17 04:51:09.504246441 (203h25m3.935326572s) [seg_pos={0 0}, upload=211594]: Done begin upload segment. Aug 17 04:51:09.504264686 (203h25m3.935344817s) [noise=true , address=104.36.82.2:4748 , piece_id=T6MOSFTUKYGVT2YSCEGXMSDXLULB6NI4RGGGB3SSARO7ALNZJN5Q, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504308099 (203h25m3.93538823s) [noise=true , address=104.36.82.19:4436 , piece_id=VN6NIXXOT6KUQE3JU6X35TPYNCI5K4KOSBLVXMOKSP6N2SYRFM6Q, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504588896 (203h25m3.935669017s) [noise=true , address=104.36.82.58:4295 , piece_id=3LNVXIW5IU2BQZRH6QAVV7A2PJ3L3YSGNOXBMIYOJMFBVHJZUPYA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504650284 (203h25m3.935730405s) [noise=true , address=64.49.13.169:4706 , piece_id=Y4SS3PCBRVCQS5N5N34Y6PGUJ5FPWOCHHO5KMU2PJ5CUXWTHW7RQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504844445 (203h25m3.935924566s) [noise=true , address=45.91.116.12:4228 , piece_id=X5VGNN2WVGEW3E4MPUO5MQWZ7L2JMB345IQUUTBI4DYW5HESK33A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504957571 (203h25m3.936037702s) [noise=true , address=104.36.82.23:4475 , piece_id=TYHG2VUDQ4RRMFLUOLV3U73G7JXA6ZRC757TD34Q2MZTJNRMUIZA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505108992 (203h25m3.936189123s) [noise=true , address=104.36.82.59:4331 , piece_id=SPNJSZRY7LNIS7ZCHJLOOWKVZU7AP3CODFM73CGOLQZU7HUCT2IQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505263607 (203h25m3.936343738s) [noise=true , address=66.23.193.175:4553 , piece_id=BXVNQMDXCQPEUSLIZ24VI22F7FOOJN2MEGJU6UI2SDQIZJSFQJ2A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505341626 (203h25m3.936421757s) [noise=true , address=77.90.31.10:4523 , piece_id=22C56K4AOQCL5N6WR7QECVAAXHOS3OUNMGWCRFHP5GKSTHIT7OLA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505565765 (203h25m3.936645896s) [noise=true , address=104.36.82.4:4836 , piece_id=JJWCMFYVQUU6WGEVBEJS4JPZQNNZERI5BVTBBAQQUZV3WD63NGIA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505597326 (203h25m3.936677457s) [noise=true , address=77.90.31.11:4563 , piece_id=QDC72JPWTU4LH5URAU44ECPV72L4NQPWGDDOHHVYSGNJTFS7C6XQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505853546 (203h25m3.936933677s) [noise=true , address=104.36.82.18:4396 , piece_id=3CORBTMNXLB5AG6ZGAPLPBGGFEQ4SSMJHRGXPESWTJD5YPYVUKFA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505894053 (203h25m3.936974184s) [noise=true , address=104.36.82.44:4896 , piece_id=2DQKKZFJX4N6W5UOOYX4QWSZ3GAFBMQIIDCKU3N3N5MLM4Q7T2AQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506090339 (203h25m3.937170471s) [noise=true , address=77.90.31.13:4658 , piece_id=XONXAMV3VAHHSHNIDMAIMMD6KOCKLYJQAUTNVMSUXB6ZCWP5THDA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506165653 (203h25m3.937245784s) [noise=true , address=104.36.82.5:4850 , piece_id=6ZYI2JS5HMQKTUM2XR4ZTSJMC3MHVILV4YOQ7ANA5PX3FDJXJ2DA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506275653 (203h25m3.937355774s) [noise=true , address=45.91.116.14:4199 , piece_id=HOTL5QUTJLBBZNF6A4WPRKMQXB72I3TUVSDYUVWMU7G4ZIXOIK3A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506396365 (203h25m3.937476496s) [noise=true , address=104.36.82.34:4565 , piece_id=LUXKT7N4OUT4UJA345KTE2T3QUD3LEGLUQKQTDZADYBOGE2Z5XNQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506467330 (203h25m3.937547461s) [noise=true , address=104.36.82.60:4354 , piece_id=M3INHRZ2XHGLNFSI7OHKHKNCVTSOWTSCJJCGHTTYCAJGFLDWLL7A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:10.617749206 (203h25m5.048829337s) [upload=211594]: Next returned nil segment Aug 17 04:51:10.617765637 (203h25m5.048845758s) [upload=211594]: Waiting for error group managing segments... Aug 17 04:51:10.899081476 (203h25m5.330161607s) [noise=true , address=64.49.13.169:4706 , piece_id=Y4SS3PCBRVCQS5N5N34Y6PGUJ5FPWOCHHO5KMU2PJ5CUXWTHW7RQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:10.959773754 (203h25m5.390853875s) [noise=true , address=66.23.193.175:4553 , piece_id=BXVNQMDXCQPEUSLIZ24VI22F7FOOJN2MEGJU6UI2SDQIZJSFQJ2A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.387426163 (203h25m5.818506294s) [noise=true , address=77.90.31.11:4563 , piece_id=QDC72JPWTU4LH5URAU44ECPV72L4NQPWGDDOHHVYSGNJTFS7C6XQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.392712709 (203h25m5.82379284s) [noise=true , address=45.91.116.14:4199 , piece_id=HOTL5QUTJLBBZNF6A4WPRKMQXB72I3TUVSDYUVWMU7G4ZIXOIK3A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.399286516 (203h25m5.830366637s) [noise=true , address=104.36.82.4:4836 , piece_id=JJWCMFYVQUU6WGEVBEJS4JPZQNNZERI5BVTBBAQQUZV3WD63NGIA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.412750859 (203h25m5.84383099s) [noise=true , address=104.36.82.23:4475 , piece_id=TYHG2VUDQ4RRMFLUOLV3U73G7JXA6ZRC757TD34Q2MZTJNRMUIZA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.439625902 (203h25m5.870706033s) [noise=true , address=77.90.31.10:4523 , piece_id=22C56K4AOQCL5N6WR7QECVAAXHOS3OUNMGWCRFHP5GKSTHIT7OLA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.465577498 (203h25m5.896657629s) [noise=true , address=104.36.82.34:4565 , piece_id=LUXKT7N4OUT4UJA345KTE2T3QUD3LEGLUQKQTDZADYBOGE2Z5XNQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.469847087 (203h25m5.900927218s) [noise=true , address=104.36.82.60:4354 , piece_id=M3INHRZ2XHGLNFSI7OHKHKNCVTSOWTSCJJCGHTTYCAJGFLDWLL7A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.470697434 (203h25m5.901777555s) [noise=true , address=104.36.82.5:4850 , piece_id=6ZYI2JS5HMQKTUM2XR4ZTSJMC3MHVILV4YOQ7ANA5PX3FDJXJ2DA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.470949846 (203h25m5.902029978s) [noise=true , address=104.36.82.19:4436 , piece_id=VN6NIXXOT6KUQE3JU6X35TPYNCI5K4KOSBLVXMOKSP6N2SYRFM6Q, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.486560086 (203h25m5.917640217s) [noise=true , address=104.36.82.2:4748 , piece_id=T6MOSFTUKYGVT2YSCEGXMSDXLULB6NI4RGGGB3SSARO7ALNZJN5Q, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.498376395 (203h25m5.929456526s) [noise=true , address=104.36.82.44:4896 , piece_id=2DQKKZFJX4N6W5UOOYX4QWSZ3GAFBMQIIDCKU3N3N5MLM4Q7T2AQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.512939881 (203h25m5.944020012s) [noise=true , address=45.91.116.12:4228 , piece_id=X5VGNN2WVGEW3E4MPUO5MQWZ7L2JMB345IQUUTBI4DYW5HESK33A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.512969688 (203h25m5.944049819s) [seg_pos={0 0}, upload=211594]: Segment reached optimal threshold of 14 pieces. Aug 17 04:51:11.513156405 (203h25m5.944236536s) [noise=true , address=77.90.31.13:4658 , piece_id=XONXAMV3VAHHSHNIDMAIMMD6KOCKLYJQAUTNVMSUXB6ZCWP5THDA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129aongkQE9HMNz8vsNNA5SvKbhsHqdt9UrtSTPJJpP9mkFsboX): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513220778 (203h25m5.944300909s) [noise=true , address=104.36.82.58:4295 , piece_id=3LNVXIW5IU2BQZRH6QAVV7A2PJ3L3YSGNOXBMIYOJMFBVHJZUPYA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oLa34qwPBAx3CTvYYj7GN3oxxWGJ9dAYXfke1eUftZc6xzfqg): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513269471 (203h25m5.944349603s) [noise=true , address=104.36.82.59:4331 , piece_id=SPNJSZRY7LNIS7ZCHJLOOWKVZU7AP3CODFM73CGOLQZU7HUCT2IQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513316901 (203h25m5.944397033s) [noise=true , address=104.36.82.18:4396 , piece_id=3CORBTMNXLB5AG6ZGAPLPBGGFEQ4SSMJHRGXPESWTJD5YPYVUKFA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1k5gzJBGWhpARcfiynburgfTqJ2t7sVNs1mAkUfToMG5saxPqx): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513333754 (203h25m5.944413885s) [seg_pos={0 0}, upload=211594]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 04:51:11.513406654 (203h25m5.944486775s) [upload=211594]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 04:51:11.513424287 (203h25m5.944504418s) [upload=211594]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:51:11.809871930 (203h25m6.240952061s) [upload=211594]: Done uploading segments... Aug 17 04:51:11.809892119 (203h25m6.24097225s) [upload=211594]: Upload finished. err: <nil> Duration: 5.802208888s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [..............................X.....X.....................XX] duration: 1.822451944s FAIL (2025-08-17 04:56:51 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 04:56:50.046028598 (203h30m44.477108729s) [upload=211693]: Starting upload Aug 17 04:56:50.046078113 (203h30m44.477158244s) [upload=211693]: Done starting upload Aug 17 04:56:50.046127948 (203h30m44.477208079s) [upload=211693]: Uploading segments... Aug 17 04:56:50.046165199 (203h30m44.477245331s) [upload=211693]: Got next segment. Inline: true Aug 17 04:56:50.046169808 (203h30m44.477249929s) [upload=211693]: Waiting for error group managing segments... Aug 17 04:56:50.046201128 (203h30m44.477281249s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:56:50.046204875 (203h30m44.477285006s) [upload=211693]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:56:50.046207780 (203h30m44.477287911s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:56:50.545849463 (203h30m44.976929594s) [upload=211693]: Done uploading segments... Aug 17 04:56:50.545868880 (203h30m44.976949011s) [upload=211693]: Upload finished. err: <nil> Duration:* 1.822477432s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........XXXX..............................................] duration: 5.701345969s FAIL (2025-08-17 04:56:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 04:23:03.873697274 (202h56m58.304777405s) [upload=211112]: Starting upload Aug 17 04:23:03.873756368 (202h56m58.304836499s) [upload=211112]: Done starting upload Aug 17 04:23:03.873839848 (202h56m58.304919989s) [upload=211112]: Uploading segments... Aug 17 04:23:03.873950850 (202h56m58.305030981s) [upload=211112]: Got next segment. Inline: false Aug 17 04:23:03.873976509 (202h56m58.30505664s) [upload=211112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:03.873980757 (202h56m58.305060888s) [upload=211112]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:04.113727713 (202h56m58.544807844s) [seg_pos={0 0}, upload=211112]: Begin upload segment... Aug 17 04:23:04.113774051 (202h56m58.544854182s) [seg_pos={0 0}, upload=211112]: Done begin upload segment. Aug 17 04:23:04.113820890 (202h56m58.544901021s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.113845989 (202h56m58.54492612s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114195037 (202h56m58.545275168s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114281412 (202h56m58.545361543s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114482115 (202h56m58.545562237s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114570385 (202h56m58.545650516s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114757392 (202h56m58.545837523s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114874727 (202h56m58.545954858s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115030715 (202h56m58.546110846s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115194288 (202h56m58.546274419s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115307967 (202h56m58.546388098s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115535692 (202h56m58.546615823s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115754130 (202h56m58.546834261s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116079182 (202h56m58.547159323s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116350181 (202h56m58.547430312s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116554471 (202h56m58.547634602s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:05.868765063 (202h57m0.299845195s) [upload=211112]: Next returned nil segment Aug 17 04:23:05.868801043 (202h57m0.299881164s) [upload=211112]: Waiting for error group managing segments... Aug 17 04:23:06.147430529 (202h57m0.57851066s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.182400314 (202h57m0.613480445s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.191910532 (202h57m0.622990663s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.197391631 (202h57m0.628471782s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.198069487 (202h57m0.629149618s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.200597014 (202h57m0.631677145s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.288053251 (202h57m0.719133372s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.298810435 (202h57m0.729890566s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.301049520 (202h57m0.732129651s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.402776985 (202h57m0.833857116s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:33.223003412 (202h57m27.654083543s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: write tcp 188.166.225.170:42258->104.36.82.5:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223050702 (202h57m27.654130823s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:52308->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223090619 (202h57m27.65417075s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 188.166.225.170:53078->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223104976 (202h57m27.654185097s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:56176->104.36.82.4:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223149581 (202h57m27.654229712s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:40582->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223157536 (202h57m27.654237667s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 188.166.225.170:32954->104.36.82.9:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223177415 (202h57m27.654257536s) [seg_pos={0 0}, upload=211112]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 04:23:33.448862924 (202h57m27.879943045s) [upload=211112]: Done uploading segments... Aug 17 04:23:33.448886830 (202h57m27.879966961s) [upload=211112]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.701373822s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 10 out of 60 is failed (threshold=3) [...X..........XXXX.X.X.......X...X.................X........] duration: 5.581323249s FAIL (2025-08-17 04:56:34 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 04:51:09.056938814 (203h25m3.488018935s) [upload=211594]: Starting upload Aug 17 04:51:09.057017635 (203h25m3.488097766s) [upload=211594]: Done starting upload Aug 17 04:51:09.057190195 (203h25m3.488270326s) [upload=211594]: Uploading segments... Aug 17 04:51:09.057308371 (203h25m3.488388502s) [upload=211594]: Got next segment. Inline: false Aug 17 04:51:09.057342866 (203h25m3.488422997s) [upload=211594]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:51:09.057347857 (203h25m3.488427978s) [upload=211594]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:51:09.504198039 (203h25m3.93527816s) [seg_pos={0 0}, upload=211594]: Begin upload segment... Aug 17 04:51:09.504246441 (203h25m3.935326572s) [seg_pos={0 0}, upload=211594]: Done begin upload segment. Aug 17 04:51:09.504264686 (203h25m3.935344817s) [noise=true , address=104.36.82.2:4748 , piece_id=T6MOSFTUKYGVT2YSCEGXMSDXLULB6NI4RGGGB3SSARO7ALNZJN5Q, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504308099 (203h25m3.93538823s) [noise=true , address=104.36.82.19:4436 , piece_id=VN6NIXXOT6KUQE3JU6X35TPYNCI5K4KOSBLVXMOKSP6N2SYRFM6Q, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504588896 (203h25m3.935669017s) [noise=true , address=104.36.82.58:4295 , piece_id=3LNVXIW5IU2BQZRH6QAVV7A2PJ3L3YSGNOXBMIYOJMFBVHJZUPYA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504650284 (203h25m3.935730405s) [noise=true , address=64.49.13.169:4706 , piece_id=Y4SS3PCBRVCQS5N5N34Y6PGUJ5FPWOCHHO5KMU2PJ5CUXWTHW7RQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504844445 (203h25m3.935924566s) [noise=true , address=45.91.116.12:4228 , piece_id=X5VGNN2WVGEW3E4MPUO5MQWZ7L2JMB345IQUUTBI4DYW5HESK33A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504957571 (203h25m3.936037702s) [noise=true , address=104.36.82.23:4475 , piece_id=TYHG2VUDQ4RRMFLUOLV3U73G7JXA6ZRC757TD34Q2MZTJNRMUIZA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505108992 (203h25m3.936189123s) [noise=true , address=104.36.82.59:4331 , piece_id=SPNJSZRY7LNIS7ZCHJLOOWKVZU7AP3CODFM73CGOLQZU7HUCT2IQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505263607 (203h25m3.936343738s) [noise=true , address=66.23.193.175:4553 , piece_id=BXVNQMDXCQPEUSLIZ24VI22F7FOOJN2MEGJU6UI2SDQIZJSFQJ2A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505341626 (203h25m3.936421757s) [noise=true , address=77.90.31.10:4523 , piece_id=22C56K4AOQCL5N6WR7QECVAAXHOS3OUNMGWCRFHP5GKSTHIT7OLA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505565765 (203h25m3.936645896s) [noise=true , address=104.36.82.4:4836 , piece_id=JJWCMFYVQUU6WGEVBEJS4JPZQNNZERI5BVTBBAQQUZV3WD63NGIA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505597326 (203h25m3.936677457s) [noise=true , address=77.90.31.11:4563 , piece_id=QDC72JPWTU4LH5URAU44ECPV72L4NQPWGDDOHHVYSGNJTFS7C6XQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505853546 (203h25m3.936933677s) [noise=true , address=104.36.82.18:4396 , piece_id=3CORBTMNXLB5AG6ZGAPLPBGGFEQ4SSMJHRGXPESWTJD5YPYVUKFA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505894053 (203h25m3.936974184s) [noise=true , address=104.36.82.44:4896 , piece_id=2DQKKZFJX4N6W5UOOYX4QWSZ3GAFBMQIIDCKU3N3N5MLM4Q7T2AQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506090339 (203h25m3.937170471s) [noise=true , address=77.90.31.13:4658 , piece_id=XONXAMV3VAHHSHNIDMAIMMD6KOCKLYJQAUTNVMSUXB6ZCWP5THDA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506165653 (203h25m3.937245784s) [noise=true , address=104.36.82.5:4850 , piece_id=6ZYI2JS5HMQKTUM2XR4ZTSJMC3MHVILV4YOQ7ANA5PX3FDJXJ2DA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506275653 (203h25m3.937355774s) [noise=true , address=45.91.116.14:4199 , piece_id=HOTL5QUTJLBBZNF6A4WPRKMQXB72I3TUVSDYUVWMU7G4ZIXOIK3A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506396365 (203h25m3.937476496s) [noise=true , address=104.36.82.34:4565 , piece_id=LUXKT7N4OUT4UJA345KTE2T3QUD3LEGLUQKQTDZADYBOGE2Z5XNQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506467330 (203h25m3.937547461s) [noise=true , address=104.36.82.60:4354 , piece_id=M3INHRZ2XHGLNFSI7OHKHKNCVTSOWTSCJJCGHTTYCAJGFLDWLL7A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:10.617749206 (203h25m5.048829337s) [upload=211594]: Next returned nil segment Aug 17 04:51:10.617765637 (203h25m5.048845758s) [upload=211594]: Waiting for error group managing segments... Aug 17 04:51:10.899081476 (203h25m5.330161607s) [noise=true , address=64.49.13.169:4706 , piece_id=Y4SS3PCBRVCQS5N5N34Y6PGUJ5FPWOCHHO5KMU2PJ5CUXWTHW7RQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:10.959773754 (203h25m5.390853875s) [noise=true , address=66.23.193.175:4553 , piece_id=BXVNQMDXCQPEUSLIZ24VI22F7FOOJN2MEGJU6UI2SDQIZJSFQJ2A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.387426163 (203h25m5.818506294s) [noise=true , address=77.90.31.11:4563 , piece_id=QDC72JPWTU4LH5URAU44ECPV72L4NQPWGDDOHHVYSGNJTFS7C6XQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.392712709 (203h25m5.82379284s) [noise=true , address=45.91.116.14:4199 , piece_id=HOTL5QUTJLBBZNF6A4WPRKMQXB72I3TUVSDYUVWMU7G4ZIXOIK3A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.399286516 (203h25m5.830366637s) [noise=true , address=104.36.82.4:4836 , piece_id=JJWCMFYVQUU6WGEVBEJS4JPZQNNZERI5BVTBBAQQUZV3WD63NGIA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.412750859 (203h25m5.84383099s) [noise=true , address=104.36.82.23:4475 , piece_id=TYHG2VUDQ4RRMFLUOLV3U73G7JXA6ZRC757TD34Q2MZTJNRMUIZA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.439625902 (203h25m5.870706033s) [noise=true , address=77.90.31.10:4523 , piece_id=22C56K4AOQCL5N6WR7QECVAAXHOS3OUNMGWCRFHP5GKSTHIT7OLA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.465577498 (203h25m5.896657629s) [noise=true , address=104.36.82.34:4565 , piece_id=LUXKT7N4OUT4UJA345KTE2T3QUD3LEGLUQKQTDZADYBOGE2Z5XNQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.469847087 (203h25m5.900927218s) [noise=true , address=104.36.82.60:4354 , piece_id=M3INHRZ2XHGLNFSI7OHKHKNCVTSOWTSCJJCGHTTYCAJGFLDWLL7A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.470697434 (203h25m5.901777555s) [noise=true , address=104.36.82.5:4850 , piece_id=6ZYI2JS5HMQKTUM2XR4ZTSJMC3MHVILV4YOQ7ANA5PX3FDJXJ2DA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.470949846 (203h25m5.902029978s) [noise=true , address=104.36.82.19:4436 , piece_id=VN6NIXXOT6KUQE3JU6X35TPYNCI5K4KOSBLVXMOKSP6N2SYRFM6Q, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.486560086 (203h25m5.917640217s) [noise=true , address=104.36.82.2:4748 , piece_id=T6MOSFTUKYGVT2YSCEGXMSDXLULB6NI4RGGGB3SSARO7ALNZJN5Q, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.498376395 (203h25m5.929456526s) [noise=true , address=104.36.82.44:4896 , piece_id=2DQKKZFJX4N6W5UOOYX4QWSZ3GAFBMQIIDCKU3N3N5MLM4Q7T2AQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.512939881 (203h25m5.944020012s) [noise=true , address=45.91.116.12:4228 , piece_id=X5VGNN2WVGEW3E4MPUO5MQWZ7L2JMB345IQUUTBI4DYW5HESK33A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.512969688 (203h25m5.944049819s) [seg_pos={0 0}, upload=211594]: Segment reached optimal threshold of 14 pieces. Aug 17 04:51:11.513156405 (203h25m5.944236536s) [noise=true , address=77.90.31.13:4658 , piece_id=XONXAMV3VAHHSHNIDMAIMMD6KOCKLYJQAUTNVMSUXB6ZCWP5THDA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129aongkQE9HMNz8vsNNA5SvKbhsHqdt9UrtSTPJJpP9mkFsboX): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513220778 (203h25m5.944300909s) [noise=true , address=104.36.82.58:4295 , piece_id=3LNVXIW5IU2BQZRH6QAVV7A2PJ3L3YSGNOXBMIYOJMFBVHJZUPYA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oLa34qwPBAx3CTvYYj7GN3oxxWGJ9dAYXfke1eUftZc6xzfqg): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513269471 (203h25m5.944349603s) [noise=true , address=104.36.82.59:4331 , piece_id=SPNJSZRY7LNIS7ZCHJLOOWKVZU7AP3CODFM73CGOLQZU7HUCT2IQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513316901 (203h25m5.944397033s) [noise=true , address=104.36.82.18:4396 , piece_id=3CORBTMNXLB5AG6ZGAPLPBGGFEQ4SSMJHRGXPESWTJD5YPYVUKFA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1k5gzJBGWhpARcfiynburgfTqJ2t7sVNs1mAkUfToMG5saxPqx): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513333754 (203h25m5.944413885s) [seg_pos={0 0}, upload=211594]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 04:51:11.513406654 (203h25m5.944486775s) [upload=211594]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 04:51:11.513424287 (203h25m5.944504418s) [upload=211594]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:51:11.809871930 (203h25m6.240952061s) [upload=211594]: Done uploading segments... Aug 17 04:51:11.809892119 (203h25m6.24097225s) [upload=211594]: Upload finished. err: <nil> Duration: 5.581348096s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...............................X.....X.....................X] duration: 1.72979253s FAIL (2025-08-17 04:56:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 12780af4a84a6d03e400f532f6bfc24c78168f4af4e996014ef8fdc6e341c0c0 != before 0e8b43c182dd1ccf1a59dca33572eb7ae940f15e89d1af7e682dbd5efa0f574b ======LOGS====== Aug 17 04:56:16.153469894 (203h30m10.584550015s) [upload=211685]: Starting upload Aug 17 04:56:16.153539278 (203h30m10.584619409s) [upload=211685]: Done starting upload Aug 17 04:56:16.153658475 (203h30m10.584738626s) [upload=211685]: Uploading segments... Aug 17 04:56:16.153762615 (203h30m10.584842746s) [upload=211685]: Got next segment. Inline: true Aug 17 04:56:16.153772353 (203h30m10.584852504s) [upload=211685]: Waiting for error group managing segments... Aug 17 04:56:16.153863428 (203h30m10.584943569s) [upload=211685]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:56:16.153870631 (203h30m10.584950763s) [upload=211685]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:56:16.153874519 (203h30m10.58495465s) [upload=211685]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:56:16.577298788 (203h30m11.008378909s) [upload=211685]: Done uploading segments... Aug 17 04:56:16.577317924 (203h30m11.008398055s) [upload=211685]: Upload finished. err: <nil> Duration:* 1.72981889s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........XXXX..............................................] duration: 6.250133118s FAIL (2025-08-17 04:56:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.250148558s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........XXXX.............................................] duration: 5.84780923s FAIL (2025-08-17 04:56:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 04:23:03.873697274 (202h56m58.304777405s) [upload=211112]: Starting upload Aug 17 04:23:03.873756368 (202h56m58.304836499s) [upload=211112]: Done starting upload Aug 17 04:23:03.873839848 (202h56m58.304919989s) [upload=211112]: Uploading segments... Aug 17 04:23:03.873950850 (202h56m58.305030981s) [upload=211112]: Got next segment. Inline: false Aug 17 04:23:03.873976509 (202h56m58.30505664s) [upload=211112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:03.873980757 (202h56m58.305060888s) [upload=211112]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:04.113727713 (202h56m58.544807844s) [seg_pos={0 0}, upload=211112]: Begin upload segment... Aug 17 04:23:04.113774051 (202h56m58.544854182s) [seg_pos={0 0}, upload=211112]: Done begin upload segment. Aug 17 04:23:04.113820890 (202h56m58.544901021s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.113845989 (202h56m58.54492612s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114195037 (202h56m58.545275168s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114281412 (202h56m58.545361543s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114482115 (202h56m58.545562237s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114570385 (202h56m58.545650516s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114757392 (202h56m58.545837523s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114874727 (202h56m58.545954858s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115030715 (202h56m58.546110846s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115194288 (202h56m58.546274419s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115307967 (202h56m58.546388098s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115535692 (202h56m58.546615823s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115754130 (202h56m58.546834261s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116079182 (202h56m58.547159323s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116350181 (202h56m58.547430312s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116554471 (202h56m58.547634602s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:05.868765063 (202h57m0.299845195s) [upload=211112]: Next returned nil segment Aug 17 04:23:05.868801043 (202h57m0.299881164s) [upload=211112]: Waiting for error group managing segments... Aug 17 04:23:06.147430529 (202h57m0.57851066s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.182400314 (202h57m0.613480445s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.191910532 (202h57m0.622990663s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.197391631 (202h57m0.628471782s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.198069487 (202h57m0.629149618s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.200597014 (202h57m0.631677145s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.288053251 (202h57m0.719133372s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.298810435 (202h57m0.729890566s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.301049520 (202h57m0.732129651s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.402776985 (202h57m0.833857116s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:33.223003412 (202h57m27.654083543s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: write tcp 188.166.225.170:42258->104.36.82.5:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223050702 (202h57m27.654130823s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:52308->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223090619 (202h57m27.65417075s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 188.166.225.170:53078->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223104976 (202h57m27.654185097s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:56176->104.36.82.4:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223149581 (202h57m27.654229712s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:40582->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223157536 (202h57m27.654237667s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 188.166.225.170:32954->104.36.82.9:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223177415 (202h57m27.654257536s) [seg_pos={0 0}, upload=211112]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 04:23:33.448862924 (202h57m27.879943045s) [upload=211112]: Done uploading segments... Aug 17 04:23:33.448886830 (202h57m27.879966961s) [upload=211112]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.847831433s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 10 out of 60 is failed (threshold=3) [....X..........XXXX.X.X.......X...X.................X.......] duration: 5.635901129s FAIL (2025-08-17 04:56:06 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 04:51:09.056938814 (203h25m3.488018935s) [upload=211594]: Starting upload Aug 17 04:51:09.057017635 (203h25m3.488097766s) [upload=211594]: Done starting upload Aug 17 04:51:09.057190195 (203h25m3.488270326s) [upload=211594]: Uploading segments... Aug 17 04:51:09.057308371 (203h25m3.488388502s) [upload=211594]: Got next segment. Inline: false Aug 17 04:51:09.057342866 (203h25m3.488422997s) [upload=211594]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:51:09.057347857 (203h25m3.488427978s) [upload=211594]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:51:09.504198039 (203h25m3.93527816s) [seg_pos={0 0}, upload=211594]: Begin upload segment... Aug 17 04:51:09.504246441 (203h25m3.935326572s) [seg_pos={0 0}, upload=211594]: Done begin upload segment. Aug 17 04:51:09.504264686 (203h25m3.935344817s) [noise=true , address=104.36.82.2:4748 , piece_id=T6MOSFTUKYGVT2YSCEGXMSDXLULB6NI4RGGGB3SSARO7ALNZJN5Q, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504308099 (203h25m3.93538823s) [noise=true , address=104.36.82.19:4436 , piece_id=VN6NIXXOT6KUQE3JU6X35TPYNCI5K4KOSBLVXMOKSP6N2SYRFM6Q, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504588896 (203h25m3.935669017s) [noise=true , address=104.36.82.58:4295 , piece_id=3LNVXIW5IU2BQZRH6QAVV7A2PJ3L3YSGNOXBMIYOJMFBVHJZUPYA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504650284 (203h25m3.935730405s) [noise=true , address=64.49.13.169:4706 , piece_id=Y4SS3PCBRVCQS5N5N34Y6PGUJ5FPWOCHHO5KMU2PJ5CUXWTHW7RQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504844445 (203h25m3.935924566s) [noise=true , address=45.91.116.12:4228 , piece_id=X5VGNN2WVGEW3E4MPUO5MQWZ7L2JMB345IQUUTBI4DYW5HESK33A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504957571 (203h25m3.936037702s) [noise=true , address=104.36.82.23:4475 , piece_id=TYHG2VUDQ4RRMFLUOLV3U73G7JXA6ZRC757TD34Q2MZTJNRMUIZA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505108992 (203h25m3.936189123s) [noise=true , address=104.36.82.59:4331 , piece_id=SPNJSZRY7LNIS7ZCHJLOOWKVZU7AP3CODFM73CGOLQZU7HUCT2IQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505263607 (203h25m3.936343738s) [noise=true , address=66.23.193.175:4553 , piece_id=BXVNQMDXCQPEUSLIZ24VI22F7FOOJN2MEGJU6UI2SDQIZJSFQJ2A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505341626 (203h25m3.936421757s) [noise=true , address=77.90.31.10:4523 , piece_id=22C56K4AOQCL5N6WR7QECVAAXHOS3OUNMGWCRFHP5GKSTHIT7OLA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505565765 (203h25m3.936645896s) [noise=true , address=104.36.82.4:4836 , piece_id=JJWCMFYVQUU6WGEVBEJS4JPZQNNZERI5BVTBBAQQUZV3WD63NGIA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505597326 (203h25m3.936677457s) [noise=true , address=77.90.31.11:4563 , piece_id=QDC72JPWTU4LH5URAU44ECPV72L4NQPWGDDOHHVYSGNJTFS7C6XQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505853546 (203h25m3.936933677s) [noise=true , address=104.36.82.18:4396 , piece_id=3CORBTMNXLB5AG6ZGAPLPBGGFEQ4SSMJHRGXPESWTJD5YPYVUKFA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505894053 (203h25m3.936974184s) [noise=true , address=104.36.82.44:4896 , piece_id=2DQKKZFJX4N6W5UOOYX4QWSZ3GAFBMQIIDCKU3N3N5MLM4Q7T2AQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506090339 (203h25m3.937170471s) [noise=true , address=77.90.31.13:4658 , piece_id=XONXAMV3VAHHSHNIDMAIMMD6KOCKLYJQAUTNVMSUXB6ZCWP5THDA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506165653 (203h25m3.937245784s) [noise=true , address=104.36.82.5:4850 , piece_id=6ZYI2JS5HMQKTUM2XR4ZTSJMC3MHVILV4YOQ7ANA5PX3FDJXJ2DA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506275653 (203h25m3.937355774s) [noise=true , address=45.91.116.14:4199 , piece_id=HOTL5QUTJLBBZNF6A4WPRKMQXB72I3TUVSDYUVWMU7G4ZIXOIK3A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506396365 (203h25m3.937476496s) [noise=true , address=104.36.82.34:4565 , piece_id=LUXKT7N4OUT4UJA345KTE2T3QUD3LEGLUQKQTDZADYBOGE2Z5XNQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506467330 (203h25m3.937547461s) [noise=true , address=104.36.82.60:4354 , piece_id=M3INHRZ2XHGLNFSI7OHKHKNCVTSOWTSCJJCGHTTYCAJGFLDWLL7A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:10.617749206 (203h25m5.048829337s) [upload=211594]: Next returned nil segment Aug 17 04:51:10.617765637 (203h25m5.048845758s) [upload=211594]: Waiting for error group managing segments... Aug 17 04:51:10.899081476 (203h25m5.330161607s) [noise=true , address=64.49.13.169:4706 , piece_id=Y4SS3PCBRVCQS5N5N34Y6PGUJ5FPWOCHHO5KMU2PJ5CUXWTHW7RQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:10.959773754 (203h25m5.390853875s) [noise=true , address=66.23.193.175:4553 , piece_id=BXVNQMDXCQPEUSLIZ24VI22F7FOOJN2MEGJU6UI2SDQIZJSFQJ2A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.387426163 (203h25m5.818506294s) [noise=true , address=77.90.31.11:4563 , piece_id=QDC72JPWTU4LH5URAU44ECPV72L4NQPWGDDOHHVYSGNJTFS7C6XQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.392712709 (203h25m5.82379284s) [noise=true , address=45.91.116.14:4199 , piece_id=HOTL5QUTJLBBZNF6A4WPRKMQXB72I3TUVSDYUVWMU7G4ZIXOIK3A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.399286516 (203h25m5.830366637s) [noise=true , address=104.36.82.4:4836 , piece_id=JJWCMFYVQUU6WGEVBEJS4JPZQNNZERI5BVTBBAQQUZV3WD63NGIA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.412750859 (203h25m5.84383099s) [noise=true , address=104.36.82.23:4475 , piece_id=TYHG2VUDQ4RRMFLUOLV3U73G7JXA6ZRC757TD34Q2MZTJNRMUIZA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.439625902 (203h25m5.870706033s) [noise=true , address=77.90.31.10:4523 , piece_id=22C56K4AOQCL5N6WR7QECVAAXHOS3OUNMGWCRFHP5GKSTHIT7OLA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.465577498 (203h25m5.896657629s) [noise=true , address=104.36.82.34:4565 , piece_id=LUXKT7N4OUT4UJA345KTE2T3QUD3LEGLUQKQTDZADYBOGE2Z5XNQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.469847087 (203h25m5.900927218s) [noise=true , address=104.36.82.60:4354 , piece_id=M3INHRZ2XHGLNFSI7OHKHKNCVTSOWTSCJJCGHTTYCAJGFLDWLL7A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.470697434 (203h25m5.901777555s) [noise=true , address=104.36.82.5:4850 , piece_id=6ZYI2JS5HMQKTUM2XR4ZTSJMC3MHVILV4YOQ7ANA5PX3FDJXJ2DA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.470949846 (203h25m5.902029978s) [noise=true , address=104.36.82.19:4436 , piece_id=VN6NIXXOT6KUQE3JU6X35TPYNCI5K4KOSBLVXMOKSP6N2SYRFM6Q, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.486560086 (203h25m5.917640217s) [noise=true , address=104.36.82.2:4748 , piece_id=T6MOSFTUKYGVT2YSCEGXMSDXLULB6NI4RGGGB3SSARO7ALNZJN5Q, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.498376395 (203h25m5.929456526s) [noise=true , address=104.36.82.44:4896 , piece_id=2DQKKZFJX4N6W5UOOYX4QWSZ3GAFBMQIIDCKU3N3N5MLM4Q7T2AQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.512939881 (203h25m5.944020012s) [noise=true , address=45.91.116.12:4228 , piece_id=X5VGNN2WVGEW3E4MPUO5MQWZ7L2JMB345IQUUTBI4DYW5HESK33A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.512969688 (203h25m5.944049819s) [seg_pos={0 0}, upload=211594]: Segment reached optimal threshold of 14 pieces. Aug 17 04:51:11.513156405 (203h25m5.944236536s) [noise=true , address=77.90.31.13:4658 , piece_id=XONXAMV3VAHHSHNIDMAIMMD6KOCKLYJQAUTNVMSUXB6ZCWP5THDA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129aongkQE9HMNz8vsNNA5SvKbhsHqdt9UrtSTPJJpP9mkFsboX): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513220778 (203h25m5.944300909s) [noise=true , address=104.36.82.58:4295 , piece_id=3LNVXIW5IU2BQZRH6QAVV7A2PJ3L3YSGNOXBMIYOJMFBVHJZUPYA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oLa34qwPBAx3CTvYYj7GN3oxxWGJ9dAYXfke1eUftZc6xzfqg): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513269471 (203h25m5.944349603s) [noise=true , address=104.36.82.59:4331 , piece_id=SPNJSZRY7LNIS7ZCHJLOOWKVZU7AP3CODFM73CGOLQZU7HUCT2IQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513316901 (203h25m5.944397033s) [noise=true , address=104.36.82.18:4396 , piece_id=3CORBTMNXLB5AG6ZGAPLPBGGFEQ4SSMJHRGXPESWTJD5YPYVUKFA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1k5gzJBGWhpARcfiynburgfTqJ2t7sVNs1mAkUfToMG5saxPqx): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513333754 (203h25m5.944413885s) [seg_pos={0 0}, upload=211594]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 04:51:11.513406654 (203h25m5.944486775s) [upload=211594]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 04:51:11.513424287 (203h25m5.944504418s) [upload=211594]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:51:11.809871930 (203h25m6.240952061s) [upload=211594]: Done uploading segments... Aug 17 04:51:11.809892119 (203h25m6.24097225s) [upload=211594]: Upload finished. err: <nil> Duration: 5.635925115s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [................XX..........................................] duration: 6.462594329s FAIL (2025-08-17 04:56:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.462618877s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........XXX..............................................] duration: 5.681425725s FAIL (2025-08-17 04:55:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== g_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761278730 (202h56m21.192358861s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761661883 (202h56m21.192742014s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761716548 (202h56m21.192796679s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761972016 (202h56m21.193052147s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762254878 (202h56m21.193335009s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762523682 (202h56m21.193603803s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762864294 (202h56m21.193944415s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763179537 (202h56m21.194259668s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763478499 (202h56m21.19455864s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763757683 (202h56m21.194837814s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763988195 (202h56m21.195068326s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764273320 (202h56m21.195353451s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764555100 (202h56m21.195635221s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764757416 (202h56m21.195837547s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.765868993 (202h56m21.196949124s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.767152809 (202h56m21.19823294s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.768577886 (202h56m21.199658017s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:27.934309011 (202h56m22.365389142s) [upload=211101]: Next returned nil segment Aug 17 04:22:27.934342545 (202h56m22.365422666s) [upload=211101]: Waiting for error group managing segments... Aug 17 04:22:28.231985083 (202h56m22.663065214s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.670701256 (202h56m23.101781427s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.671952431 (202h56m23.103032562s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.698468278 (202h56m23.129548409s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.720171187 (202h56m23.151251318s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.731591429 (202h56m23.16267156s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.798148986 (202h56m23.229229117s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.804849037 (202h56m23.235929168s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.833066822 (202h56m23.264147013s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.846684710 (202h56m23.277764831s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:23:25.582810947 (202h57m20.013891078s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12TU4zvu38mF1fNSergCtMj2cK9eu57Rdfpt5MVk3tc4MZF6dNr, address:104.36.82.34:4558): protocol: rpc: dial tcp 104.36.82.34:4558: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582856475 (202h57m20.013936606s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12D9nh4XFmsjjFcZEMjaL4uw4Whsa8nfMrAsYPu6NVpsxgb3jvf, address:104.36.82.35:4594): protocol: rpc: dial tcp 104.36.82.35:4594: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582973238 (202h57m20.014053369s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1WvyEw8XevCiQ8yRgzyMo7WTVzpEJ158ErKfAeV4dSTBkS1oRF, address:104.36.82.19:4436): protocol: rpc: dial tcp 104.36.82.19:4436: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583033393 (202h57m20.014113524s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:104.36.82.58:4283): protocol: rpc: dial tcp 104.36.82.58:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583069241 (202h57m20.014149362s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:126Z6tvp49dLLdj3T5snJc4tBVGzcX5UBaGafFN3PkhF92vxpfs, address:104.36.82.50:4047): protocol: rpc: dial tcp 104.36.82.50:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583076716 (202h57m20.014156837s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1zzW2z6noGQT6xdSmwSQBNYwip6gZjnmqZg7qdb6BP3nHyCLFz, address:104.36.82.39:4753): protocol: rpc: dial tcp 104.36.82.39:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583107535 (202h57m20.014187666s) [noise=true , address=104.36.82.23:4474 , piece_id=RITPHITVW655UQETT6UKS7BZO2S3F4P2MJIO4REOUKYYP4E2CDVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 188.166.225.170:36642->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583165355 (202h57m20.014245487s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh, address:104.36.82.5:4855): protocol: write tcp 188.166.225.170:47052->104.36.82.5:4855: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583194000 (202h57m20.014274131s) [seg_pos={0 0}, upload=211101]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:25.810738626 (202h57m20.241818757s) [upload=211101]: Done uploading segments... Aug 17 04:23:25.810761991 (202h57m20.241842122s) [upload=211101]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.681450142s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........XXXX.............................................] duration: 6.192007498s FAIL (2025-08-17 04:55:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.192029211s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 10 out of 60 is failed (threshold=3) [.....X..........XXXX.X.X.......X...X.................X......] duration: 5.51885477s FAIL (2025-08-17 04:55:29 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 04:51:09.056938814 (203h25m3.488018935s) [upload=211594]: Starting upload Aug 17 04:51:09.057017635 (203h25m3.488097766s) [upload=211594]: Done starting upload Aug 17 04:51:09.057190195 (203h25m3.488270326s) [upload=211594]: Uploading segments... Aug 17 04:51:09.057308371 (203h25m3.488388502s) [upload=211594]: Got next segment. Inline: false Aug 17 04:51:09.057342866 (203h25m3.488422997s) [upload=211594]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:51:09.057347857 (203h25m3.488427978s) [upload=211594]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:51:09.504198039 (203h25m3.93527816s) [seg_pos={0 0}, upload=211594]: Begin upload segment... Aug 17 04:51:09.504246441 (203h25m3.935326572s) [seg_pos={0 0}, upload=211594]: Done begin upload segment. Aug 17 04:51:09.504264686 (203h25m3.935344817s) [noise=true , address=104.36.82.2:4748 , piece_id=T6MOSFTUKYGVT2YSCEGXMSDXLULB6NI4RGGGB3SSARO7ALNZJN5Q, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504308099 (203h25m3.93538823s) [noise=true , address=104.36.82.19:4436 , piece_id=VN6NIXXOT6KUQE3JU6X35TPYNCI5K4KOSBLVXMOKSP6N2SYRFM6Q, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504588896 (203h25m3.935669017s) [noise=true , address=104.36.82.58:4295 , piece_id=3LNVXIW5IU2BQZRH6QAVV7A2PJ3L3YSGNOXBMIYOJMFBVHJZUPYA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504650284 (203h25m3.935730405s) [noise=true , address=64.49.13.169:4706 , piece_id=Y4SS3PCBRVCQS5N5N34Y6PGUJ5FPWOCHHO5KMU2PJ5CUXWTHW7RQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504844445 (203h25m3.935924566s) [noise=true , address=45.91.116.12:4228 , piece_id=X5VGNN2WVGEW3E4MPUO5MQWZ7L2JMB345IQUUTBI4DYW5HESK33A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504957571 (203h25m3.936037702s) [noise=true , address=104.36.82.23:4475 , piece_id=TYHG2VUDQ4RRMFLUOLV3U73G7JXA6ZRC757TD34Q2MZTJNRMUIZA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505108992 (203h25m3.936189123s) [noise=true , address=104.36.82.59:4331 , piece_id=SPNJSZRY7LNIS7ZCHJLOOWKVZU7AP3CODFM73CGOLQZU7HUCT2IQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505263607 (203h25m3.936343738s) [noise=true , address=66.23.193.175:4553 , piece_id=BXVNQMDXCQPEUSLIZ24VI22F7FOOJN2MEGJU6UI2SDQIZJSFQJ2A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505341626 (203h25m3.936421757s) [noise=true , address=77.90.31.10:4523 , piece_id=22C56K4AOQCL5N6WR7QECVAAXHOS3OUNMGWCRFHP5GKSTHIT7OLA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505565765 (203h25m3.936645896s) [noise=true , address=104.36.82.4:4836 , piece_id=JJWCMFYVQUU6WGEVBEJS4JPZQNNZERI5BVTBBAQQUZV3WD63NGIA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505597326 (203h25m3.936677457s) [noise=true , address=77.90.31.11:4563 , piece_id=QDC72JPWTU4LH5URAU44ECPV72L4NQPWGDDOHHVYSGNJTFS7C6XQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505853546 (203h25m3.936933677s) [noise=true , address=104.36.82.18:4396 , piece_id=3CORBTMNXLB5AG6ZGAPLPBGGFEQ4SSMJHRGXPESWTJD5YPYVUKFA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505894053 (203h25m3.936974184s) [noise=true , address=104.36.82.44:4896 , piece_id=2DQKKZFJX4N6W5UOOYX4QWSZ3GAFBMQIIDCKU3N3N5MLM4Q7T2AQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506090339 (203h25m3.937170471s) [noise=true , address=77.90.31.13:4658 , piece_id=XONXAMV3VAHHSHNIDMAIMMD6KOCKLYJQAUTNVMSUXB6ZCWP5THDA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506165653 (203h25m3.937245784s) [noise=true , address=104.36.82.5:4850 , piece_id=6ZYI2JS5HMQKTUM2XR4ZTSJMC3MHVILV4YOQ7ANA5PX3FDJXJ2DA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506275653 (203h25m3.937355774s) [noise=true , address=45.91.116.14:4199 , piece_id=HOTL5QUTJLBBZNF6A4WPRKMQXB72I3TUVSDYUVWMU7G4ZIXOIK3A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506396365 (203h25m3.937476496s) [noise=true , address=104.36.82.34:4565 , piece_id=LUXKT7N4OUT4UJA345KTE2T3QUD3LEGLUQKQTDZADYBOGE2Z5XNQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506467330 (203h25m3.937547461s) [noise=true , address=104.36.82.60:4354 , piece_id=M3INHRZ2XHGLNFSI7OHKHKNCVTSOWTSCJJCGHTTYCAJGFLDWLL7A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:10.617749206 (203h25m5.048829337s) [upload=211594]: Next returned nil segment Aug 17 04:51:10.617765637 (203h25m5.048845758s) [upload=211594]: Waiting for error group managing segments... Aug 17 04:51:10.899081476 (203h25m5.330161607s) [noise=true , address=64.49.13.169:4706 , piece_id=Y4SS3PCBRVCQS5N5N34Y6PGUJ5FPWOCHHO5KMU2PJ5CUXWTHW7RQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:10.959773754 (203h25m5.390853875s) [noise=true , address=66.23.193.175:4553 , piece_id=BXVNQMDXCQPEUSLIZ24VI22F7FOOJN2MEGJU6UI2SDQIZJSFQJ2A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.387426163 (203h25m5.818506294s) [noise=true , address=77.90.31.11:4563 , piece_id=QDC72JPWTU4LH5URAU44ECPV72L4NQPWGDDOHHVYSGNJTFS7C6XQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.392712709 (203h25m5.82379284s) [noise=true , address=45.91.116.14:4199 , piece_id=HOTL5QUTJLBBZNF6A4WPRKMQXB72I3TUVSDYUVWMU7G4ZIXOIK3A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.399286516 (203h25m5.830366637s) [noise=true , address=104.36.82.4:4836 , piece_id=JJWCMFYVQUU6WGEVBEJS4JPZQNNZERI5BVTBBAQQUZV3WD63NGIA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.412750859 (203h25m5.84383099s) [noise=true , address=104.36.82.23:4475 , piece_id=TYHG2VUDQ4RRMFLUOLV3U73G7JXA6ZRC757TD34Q2MZTJNRMUIZA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.439625902 (203h25m5.870706033s) [noise=true , address=77.90.31.10:4523 , piece_id=22C56K4AOQCL5N6WR7QECVAAXHOS3OUNMGWCRFHP5GKSTHIT7OLA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.465577498 (203h25m5.896657629s) [noise=true , address=104.36.82.34:4565 , piece_id=LUXKT7N4OUT4UJA345KTE2T3QUD3LEGLUQKQTDZADYBOGE2Z5XNQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.469847087 (203h25m5.900927218s) [noise=true , address=104.36.82.60:4354 , piece_id=M3INHRZ2XHGLNFSI7OHKHKNCVTSOWTSCJJCGHTTYCAJGFLDWLL7A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.470697434 (203h25m5.901777555s) [noise=true , address=104.36.82.5:4850 , piece_id=6ZYI2JS5HMQKTUM2XR4ZTSJMC3MHVILV4YOQ7ANA5PX3FDJXJ2DA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.470949846 (203h25m5.902029978s) [noise=true , address=104.36.82.19:4436 , piece_id=VN6NIXXOT6KUQE3JU6X35TPYNCI5K4KOSBLVXMOKSP6N2SYRFM6Q, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.486560086 (203h25m5.917640217s) [noise=true , address=104.36.82.2:4748 , piece_id=T6MOSFTUKYGVT2YSCEGXMSDXLULB6NI4RGGGB3SSARO7ALNZJN5Q, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.498376395 (203h25m5.929456526s) [noise=true , address=104.36.82.44:4896 , piece_id=2DQKKZFJX4N6W5UOOYX4QWSZ3GAFBMQIIDCKU3N3N5MLM4Q7T2AQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.512939881 (203h25m5.944020012s) [noise=true , address=45.91.116.12:4228 , piece_id=X5VGNN2WVGEW3E4MPUO5MQWZ7L2JMB345IQUUTBI4DYW5HESK33A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.512969688 (203h25m5.944049819s) [seg_pos={0 0}, upload=211594]: Segment reached optimal threshold of 14 pieces. Aug 17 04:51:11.513156405 (203h25m5.944236536s) [noise=true , address=77.90.31.13:4658 , piece_id=XONXAMV3VAHHSHNIDMAIMMD6KOCKLYJQAUTNVMSUXB6ZCWP5THDA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129aongkQE9HMNz8vsNNA5SvKbhsHqdt9UrtSTPJJpP9mkFsboX): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513220778 (203h25m5.944300909s) [noise=true , address=104.36.82.58:4295 , piece_id=3LNVXIW5IU2BQZRH6QAVV7A2PJ3L3YSGNOXBMIYOJMFBVHJZUPYA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oLa34qwPBAx3CTvYYj7GN3oxxWGJ9dAYXfke1eUftZc6xzfqg): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513269471 (203h25m5.944349603s) [noise=true , address=104.36.82.59:4331 , piece_id=SPNJSZRY7LNIS7ZCHJLOOWKVZU7AP3CODFM73CGOLQZU7HUCT2IQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513316901 (203h25m5.944397033s) [noise=true , address=104.36.82.18:4396 , piece_id=3CORBTMNXLB5AG6ZGAPLPBGGFEQ4SSMJHRGXPESWTJD5YPYVUKFA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1k5gzJBGWhpARcfiynburgfTqJ2t7sVNs1mAkUfToMG5saxPqx): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513333754 (203h25m5.944413885s) [seg_pos={0 0}, upload=211594]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 04:51:11.513406654 (203h25m5.944486775s) [upload=211594]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 04:51:11.513424287 (203h25m5.944504418s) [upload=211594]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:51:11.809871930 (203h25m6.240952061s) [upload=211594]: Done uploading segments... Aug 17 04:51:11.809892119 (203h25m6.24097225s) [upload=211594]: Upload finished. err: <nil> Duration: 5.51888161s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [................................X.....X.....................] duration: 1.821674665s FAIL (2025-08-17 04:55:28 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 17 04:41:26.040732864 (203h15m20.471812985s) [upload=211420]: Starting upload Aug 17 04:41:26.040780384 (203h15m20.471860515s) [upload=211420]: Done starting upload Aug 17 04:41:26.040828978 (203h15m20.471909109s) [upload=211420]: Uploading segments... Aug 17 04:41:26.040863473 (203h15m20.471943604s) [upload=211420]: Got next segment. Inline: true Aug 17 04:41:26.040867290 (203h15m20.471947422s) [upload=211420]: Waiting for error group managing segments... Aug 17 04:41:26.040897880 (203h15m20.471978011s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:41:26.040901597 (203h15m20.471981728s) [upload=211420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:41:26.040904041 (203h15m20.471984162s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:41:26.462140724 (203h15m20.893220855s) [upload=211420]: Done uploading segments... Aug 17 04:41:26.462159310 (203h15m20.893239431s) [upload=211420]: Upload finished. err: <nil> Duration:* 1.821702509s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.................XX.........................................] duration: 6.730328569s FAIL (2025-08-17 04:55:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.730361532s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [............XXX.............................................] duration: 5.711431187s FAIL (2025-08-17 04:55:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== g_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761278730 (202h56m21.192358861s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761661883 (202h56m21.192742014s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761716548 (202h56m21.192796679s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761972016 (202h56m21.193052147s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762254878 (202h56m21.193335009s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762523682 (202h56m21.193603803s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762864294 (202h56m21.193944415s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763179537 (202h56m21.194259668s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763478499 (202h56m21.19455864s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763757683 (202h56m21.194837814s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763988195 (202h56m21.195068326s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764273320 (202h56m21.195353451s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764555100 (202h56m21.195635221s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764757416 (202h56m21.195837547s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.765868993 (202h56m21.196949124s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.767152809 (202h56m21.19823294s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.768577886 (202h56m21.199658017s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:27.934309011 (202h56m22.365389142s) [upload=211101]: Next returned nil segment Aug 17 04:22:27.934342545 (202h56m22.365422666s) [upload=211101]: Waiting for error group managing segments... Aug 17 04:22:28.231985083 (202h56m22.663065214s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.670701256 (202h56m23.101781427s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.671952431 (202h56m23.103032562s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.698468278 (202h56m23.129548409s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.720171187 (202h56m23.151251318s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.731591429 (202h56m23.16267156s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.798148986 (202h56m23.229229117s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.804849037 (202h56m23.235929168s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.833066822 (202h56m23.264147013s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.846684710 (202h56m23.277764831s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:23:25.582810947 (202h57m20.013891078s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12TU4zvu38mF1fNSergCtMj2cK9eu57Rdfpt5MVk3tc4MZF6dNr, address:104.36.82.34:4558): protocol: rpc: dial tcp 104.36.82.34:4558: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582856475 (202h57m20.013936606s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12D9nh4XFmsjjFcZEMjaL4uw4Whsa8nfMrAsYPu6NVpsxgb3jvf, address:104.36.82.35:4594): protocol: rpc: dial tcp 104.36.82.35:4594: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582973238 (202h57m20.014053369s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1WvyEw8XevCiQ8yRgzyMo7WTVzpEJ158ErKfAeV4dSTBkS1oRF, address:104.36.82.19:4436): protocol: rpc: dial tcp 104.36.82.19:4436: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583033393 (202h57m20.014113524s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:104.36.82.58:4283): protocol: rpc: dial tcp 104.36.82.58:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583069241 (202h57m20.014149362s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:126Z6tvp49dLLdj3T5snJc4tBVGzcX5UBaGafFN3PkhF92vxpfs, address:104.36.82.50:4047): protocol: rpc: dial tcp 104.36.82.50:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583076716 (202h57m20.014156837s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1zzW2z6noGQT6xdSmwSQBNYwip6gZjnmqZg7qdb6BP3nHyCLFz, address:104.36.82.39:4753): protocol: rpc: dial tcp 104.36.82.39:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583107535 (202h57m20.014187666s) [noise=true , address=104.36.82.23:4474 , piece_id=RITPHITVW655UQETT6UKS7BZO2S3F4P2MJIO4REOUKYYP4E2CDVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 188.166.225.170:36642->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583165355 (202h57m20.014245487s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh, address:104.36.82.5:4855): protocol: write tcp 188.166.225.170:47052->104.36.82.5:4855: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583194000 (202h57m20.014274131s) [seg_pos={0 0}, upload=211101]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:25.810738626 (202h57m20.241818757s) [upload=211101]: Done uploading segments... Aug 17 04:23:25.810761991 (202h57m20.241842122s) [upload=211101]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.711455312s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [............XXXX............................................] duration: 5.89603798s FAIL (2025-08-17 04:55:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 04:23:03.873697274 (202h56m58.304777405s) [upload=211112]: Starting upload Aug 17 04:23:03.873756368 (202h56m58.304836499s) [upload=211112]: Done starting upload Aug 17 04:23:03.873839848 (202h56m58.304919989s) [upload=211112]: Uploading segments... Aug 17 04:23:03.873950850 (202h56m58.305030981s) [upload=211112]: Got next segment. Inline: false Aug 17 04:23:03.873976509 (202h56m58.30505664s) [upload=211112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:03.873980757 (202h56m58.305060888s) [upload=211112]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:04.113727713 (202h56m58.544807844s) [seg_pos={0 0}, upload=211112]: Begin upload segment... Aug 17 04:23:04.113774051 (202h56m58.544854182s) [seg_pos={0 0}, upload=211112]: Done begin upload segment. Aug 17 04:23:04.113820890 (202h56m58.544901021s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.113845989 (202h56m58.54492612s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114195037 (202h56m58.545275168s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114281412 (202h56m58.545361543s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114482115 (202h56m58.545562237s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114570385 (202h56m58.545650516s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114757392 (202h56m58.545837523s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114874727 (202h56m58.545954858s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115030715 (202h56m58.546110846s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115194288 (202h56m58.546274419s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115307967 (202h56m58.546388098s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115535692 (202h56m58.546615823s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115754130 (202h56m58.546834261s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116079182 (202h56m58.547159323s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116350181 (202h56m58.547430312s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116554471 (202h56m58.547634602s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:05.868765063 (202h57m0.299845195s) [upload=211112]: Next returned nil segment Aug 17 04:23:05.868801043 (202h57m0.299881164s) [upload=211112]: Waiting for error group managing segments... Aug 17 04:23:06.147430529 (202h57m0.57851066s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.182400314 (202h57m0.613480445s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.191910532 (202h57m0.622990663s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.197391631 (202h57m0.628471782s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.198069487 (202h57m0.629149618s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.200597014 (202h57m0.631677145s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.288053251 (202h57m0.719133372s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.298810435 (202h57m0.729890566s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.301049520 (202h57m0.732129651s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.402776985 (202h57m0.833857116s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:33.223003412 (202h57m27.654083543s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: write tcp 188.166.225.170:42258->104.36.82.5:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223050702 (202h57m27.654130823s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:52308->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223090619 (202h57m27.65417075s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 188.166.225.170:53078->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223104976 (202h57m27.654185097s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:56176->104.36.82.4:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223149581 (202h57m27.654229712s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:40582->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223157536 (202h57m27.654237667s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 188.166.225.170:32954->104.36.82.9:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223177415 (202h57m27.654257536s) [seg_pos={0 0}, upload=211112]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 04:23:33.448862924 (202h57m27.879943045s) [upload=211112]: Done uploading segments... Aug 17 04:23:33.448886830 (202h57m27.879966961s) [upload=211112]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.896070944s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............XXXX...........................................] duration: 5.809861551s FAIL (2025-08-17 04:54:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 04:23:03.873697274 (202h56m58.304777405s) [upload=211112]: Starting upload Aug 17 04:23:03.873756368 (202h56m58.304836499s) [upload=211112]: Done starting upload Aug 17 04:23:03.873839848 (202h56m58.304919989s) [upload=211112]: Uploading segments... Aug 17 04:23:03.873950850 (202h56m58.305030981s) [upload=211112]: Got next segment. Inline: false Aug 17 04:23:03.873976509 (202h56m58.30505664s) [upload=211112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:03.873980757 (202h56m58.305060888s) [upload=211112]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:04.113727713 (202h56m58.544807844s) [seg_pos={0 0}, upload=211112]: Begin upload segment... Aug 17 04:23:04.113774051 (202h56m58.544854182s) [seg_pos={0 0}, upload=211112]: Done begin upload segment. Aug 17 04:23:04.113820890 (202h56m58.544901021s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.113845989 (202h56m58.54492612s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114195037 (202h56m58.545275168s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114281412 (202h56m58.545361543s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114482115 (202h56m58.545562237s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114570385 (202h56m58.545650516s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114757392 (202h56m58.545837523s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114874727 (202h56m58.545954858s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115030715 (202h56m58.546110846s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115194288 (202h56m58.546274419s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115307967 (202h56m58.546388098s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115535692 (202h56m58.546615823s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115754130 (202h56m58.546834261s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116079182 (202h56m58.547159323s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116350181 (202h56m58.547430312s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116554471 (202h56m58.547634602s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:05.868765063 (202h57m0.299845195s) [upload=211112]: Next returned nil segment Aug 17 04:23:05.868801043 (202h57m0.299881164s) [upload=211112]: Waiting for error group managing segments... Aug 17 04:23:06.147430529 (202h57m0.57851066s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.182400314 (202h57m0.613480445s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.191910532 (202h57m0.622990663s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.197391631 (202h57m0.628471782s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.198069487 (202h57m0.629149618s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.200597014 (202h57m0.631677145s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.288053251 (202h57m0.719133372s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.298810435 (202h57m0.729890566s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.301049520 (202h57m0.732129651s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.402776985 (202h57m0.833857116s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:33.223003412 (202h57m27.654083543s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: write tcp 188.166.225.170:42258->104.36.82.5:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223050702 (202h57m27.654130823s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:52308->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223090619 (202h57m27.65417075s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 188.166.225.170:53078->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223104976 (202h57m27.654185097s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:56176->104.36.82.4:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223149581 (202h57m27.654229712s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:40582->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223157536 (202h57m27.654237667s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 188.166.225.170:32954->104.36.82.9:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223177415 (202h57m27.654257536s) [seg_pos={0 0}, upload=211112]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 04:23:33.448862924 (202h57m27.879943045s) [upload=211112]: Done uploading segments... Aug 17 04:23:33.448886830 (202h57m27.879966961s) [upload=211112]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.809888023s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............XXX............................................] duration: 5.599131901s FAIL (2025-08-17 04:54:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== g_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761278730 (202h56m21.192358861s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761661883 (202h56m21.192742014s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761716548 (202h56m21.192796679s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761972016 (202h56m21.193052147s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762254878 (202h56m21.193335009s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762523682 (202h56m21.193603803s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762864294 (202h56m21.193944415s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763179537 (202h56m21.194259668s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763478499 (202h56m21.19455864s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763757683 (202h56m21.194837814s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763988195 (202h56m21.195068326s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764273320 (202h56m21.195353451s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764555100 (202h56m21.195635221s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764757416 (202h56m21.195837547s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.765868993 (202h56m21.196949124s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.767152809 (202h56m21.19823294s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.768577886 (202h56m21.199658017s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:27.934309011 (202h56m22.365389142s) [upload=211101]: Next returned nil segment Aug 17 04:22:27.934342545 (202h56m22.365422666s) [upload=211101]: Waiting for error group managing segments... Aug 17 04:22:28.231985083 (202h56m22.663065214s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.670701256 (202h56m23.101781427s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.671952431 (202h56m23.103032562s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.698468278 (202h56m23.129548409s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.720171187 (202h56m23.151251318s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.731591429 (202h56m23.16267156s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.798148986 (202h56m23.229229117s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.804849037 (202h56m23.235929168s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.833066822 (202h56m23.264147013s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.846684710 (202h56m23.277764831s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:23:25.582810947 (202h57m20.013891078s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12TU4zvu38mF1fNSergCtMj2cK9eu57Rdfpt5MVk3tc4MZF6dNr, address:104.36.82.34:4558): protocol: rpc: dial tcp 104.36.82.34:4558: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582856475 (202h57m20.013936606s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12D9nh4XFmsjjFcZEMjaL4uw4Whsa8nfMrAsYPu6NVpsxgb3jvf, address:104.36.82.35:4594): protocol: rpc: dial tcp 104.36.82.35:4594: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582973238 (202h57m20.014053369s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1WvyEw8XevCiQ8yRgzyMo7WTVzpEJ158ErKfAeV4dSTBkS1oRF, address:104.36.82.19:4436): protocol: rpc: dial tcp 104.36.82.19:4436: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583033393 (202h57m20.014113524s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:104.36.82.58:4283): protocol: rpc: dial tcp 104.36.82.58:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583069241 (202h57m20.014149362s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:126Z6tvp49dLLdj3T5snJc4tBVGzcX5UBaGafFN3PkhF92vxpfs, address:104.36.82.50:4047): protocol: rpc: dial tcp 104.36.82.50:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583076716 (202h57m20.014156837s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1zzW2z6noGQT6xdSmwSQBNYwip6gZjnmqZg7qdb6BP3nHyCLFz, address:104.36.82.39:4753): protocol: rpc: dial tcp 104.36.82.39:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583107535 (202h57m20.014187666s) [noise=true , address=104.36.82.23:4474 , piece_id=RITPHITVW655UQETT6UKS7BZO2S3F4P2MJIO4REOUKYYP4E2CDVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 188.166.225.170:36642->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583165355 (202h57m20.014245487s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh, address:104.36.82.5:4855): protocol: write tcp 188.166.225.170:47052->104.36.82.5:4855: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583194000 (202h57m20.014274131s) [seg_pos={0 0}, upload=211101]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:25.810738626 (202h57m20.241818757s) [upload=211101]: Done uploading segments... Aug 17 04:23:25.810761991 (202h57m20.241842122s) [upload=211101]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.599155116s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [............XXXX............................................] duration: 6.335333295s FAIL (2025-08-17 04:54:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.33535106s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..................XX........................................] duration: 6.475617492s FAIL (2025-08-17 04:54:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.475632571s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 10 out of 60 is failed (threshold=3) [......X..........XXXX.X.X.......X...X.................X.....] duration: 5.697443792s FAIL (2025-08-17 04:54:22 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 04:51:09.056938814 (203h25m3.488018935s) [upload=211594]: Starting upload Aug 17 04:51:09.057017635 (203h25m3.488097766s) [upload=211594]: Done starting upload Aug 17 04:51:09.057190195 (203h25m3.488270326s) [upload=211594]: Uploading segments... Aug 17 04:51:09.057308371 (203h25m3.488388502s) [upload=211594]: Got next segment. Inline: false Aug 17 04:51:09.057342866 (203h25m3.488422997s) [upload=211594]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:51:09.057347857 (203h25m3.488427978s) [upload=211594]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:51:09.504198039 (203h25m3.93527816s) [seg_pos={0 0}, upload=211594]: Begin upload segment... Aug 17 04:51:09.504246441 (203h25m3.935326572s) [seg_pos={0 0}, upload=211594]: Done begin upload segment. Aug 17 04:51:09.504264686 (203h25m3.935344817s) [noise=true , address=104.36.82.2:4748 , piece_id=T6MOSFTUKYGVT2YSCEGXMSDXLULB6NI4RGGGB3SSARO7ALNZJN5Q, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504308099 (203h25m3.93538823s) [noise=true , address=104.36.82.19:4436 , piece_id=VN6NIXXOT6KUQE3JU6X35TPYNCI5K4KOSBLVXMOKSP6N2SYRFM6Q, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504588896 (203h25m3.935669017s) [noise=true , address=104.36.82.58:4295 , piece_id=3LNVXIW5IU2BQZRH6QAVV7A2PJ3L3YSGNOXBMIYOJMFBVHJZUPYA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504650284 (203h25m3.935730405s) [noise=true , address=64.49.13.169:4706 , piece_id=Y4SS3PCBRVCQS5N5N34Y6PGUJ5FPWOCHHO5KMU2PJ5CUXWTHW7RQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504844445 (203h25m3.935924566s) [noise=true , address=45.91.116.12:4228 , piece_id=X5VGNN2WVGEW3E4MPUO5MQWZ7L2JMB345IQUUTBI4DYW5HESK33A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504957571 (203h25m3.936037702s) [noise=true , address=104.36.82.23:4475 , piece_id=TYHG2VUDQ4RRMFLUOLV3U73G7JXA6ZRC757TD34Q2MZTJNRMUIZA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505108992 (203h25m3.936189123s) [noise=true , address=104.36.82.59:4331 , piece_id=SPNJSZRY7LNIS7ZCHJLOOWKVZU7AP3CODFM73CGOLQZU7HUCT2IQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505263607 (203h25m3.936343738s) [noise=true , address=66.23.193.175:4553 , piece_id=BXVNQMDXCQPEUSLIZ24VI22F7FOOJN2MEGJU6UI2SDQIZJSFQJ2A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505341626 (203h25m3.936421757s) [noise=true , address=77.90.31.10:4523 , piece_id=22C56K4AOQCL5N6WR7QECVAAXHOS3OUNMGWCRFHP5GKSTHIT7OLA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505565765 (203h25m3.936645896s) [noise=true , address=104.36.82.4:4836 , piece_id=JJWCMFYVQUU6WGEVBEJS4JPZQNNZERI5BVTBBAQQUZV3WD63NGIA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505597326 (203h25m3.936677457s) [noise=true , address=77.90.31.11:4563 , piece_id=QDC72JPWTU4LH5URAU44ECPV72L4NQPWGDDOHHVYSGNJTFS7C6XQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505853546 (203h25m3.936933677s) [noise=true , address=104.36.82.18:4396 , piece_id=3CORBTMNXLB5AG6ZGAPLPBGGFEQ4SSMJHRGXPESWTJD5YPYVUKFA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505894053 (203h25m3.936974184s) [noise=true , address=104.36.82.44:4896 , piece_id=2DQKKZFJX4N6W5UOOYX4QWSZ3GAFBMQIIDCKU3N3N5MLM4Q7T2AQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506090339 (203h25m3.937170471s) [noise=true , address=77.90.31.13:4658 , piece_id=XONXAMV3VAHHSHNIDMAIMMD6KOCKLYJQAUTNVMSUXB6ZCWP5THDA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506165653 (203h25m3.937245784s) [noise=true , address=104.36.82.5:4850 , piece_id=6ZYI2JS5HMQKTUM2XR4ZTSJMC3MHVILV4YOQ7ANA5PX3FDJXJ2DA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506275653 (203h25m3.937355774s) [noise=true , address=45.91.116.14:4199 , piece_id=HOTL5QUTJLBBZNF6A4WPRKMQXB72I3TUVSDYUVWMU7G4ZIXOIK3A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506396365 (203h25m3.937476496s) [noise=true , address=104.36.82.34:4565 , piece_id=LUXKT7N4OUT4UJA345KTE2T3QUD3LEGLUQKQTDZADYBOGE2Z5XNQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506467330 (203h25m3.937547461s) [noise=true , address=104.36.82.60:4354 , piece_id=M3INHRZ2XHGLNFSI7OHKHKNCVTSOWTSCJJCGHTTYCAJGFLDWLL7A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:10.617749206 (203h25m5.048829337s) [upload=211594]: Next returned nil segment Aug 17 04:51:10.617765637 (203h25m5.048845758s) [upload=211594]: Waiting for error group managing segments... Aug 17 04:51:10.899081476 (203h25m5.330161607s) [noise=true , address=64.49.13.169:4706 , piece_id=Y4SS3PCBRVCQS5N5N34Y6PGUJ5FPWOCHHO5KMU2PJ5CUXWTHW7RQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:10.959773754 (203h25m5.390853875s) [noise=true , address=66.23.193.175:4553 , piece_id=BXVNQMDXCQPEUSLIZ24VI22F7FOOJN2MEGJU6UI2SDQIZJSFQJ2A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.387426163 (203h25m5.818506294s) [noise=true , address=77.90.31.11:4563 , piece_id=QDC72JPWTU4LH5URAU44ECPV72L4NQPWGDDOHHVYSGNJTFS7C6XQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.392712709 (203h25m5.82379284s) [noise=true , address=45.91.116.14:4199 , piece_id=HOTL5QUTJLBBZNF6A4WPRKMQXB72I3TUVSDYUVWMU7G4ZIXOIK3A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.399286516 (203h25m5.830366637s) [noise=true , address=104.36.82.4:4836 , piece_id=JJWCMFYVQUU6WGEVBEJS4JPZQNNZERI5BVTBBAQQUZV3WD63NGIA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.412750859 (203h25m5.84383099s) [noise=true , address=104.36.82.23:4475 , piece_id=TYHG2VUDQ4RRMFLUOLV3U73G7JXA6ZRC757TD34Q2MZTJNRMUIZA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.439625902 (203h25m5.870706033s) [noise=true , address=77.90.31.10:4523 , piece_id=22C56K4AOQCL5N6WR7QECVAAXHOS3OUNMGWCRFHP5GKSTHIT7OLA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.465577498 (203h25m5.896657629s) [noise=true , address=104.36.82.34:4565 , piece_id=LUXKT7N4OUT4UJA345KTE2T3QUD3LEGLUQKQTDZADYBOGE2Z5XNQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.469847087 (203h25m5.900927218s) [noise=true , address=104.36.82.60:4354 , piece_id=M3INHRZ2XHGLNFSI7OHKHKNCVTSOWTSCJJCGHTTYCAJGFLDWLL7A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.470697434 (203h25m5.901777555s) [noise=true , address=104.36.82.5:4850 , piece_id=6ZYI2JS5HMQKTUM2XR4ZTSJMC3MHVILV4YOQ7ANA5PX3FDJXJ2DA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.470949846 (203h25m5.902029978s) [noise=true , address=104.36.82.19:4436 , piece_id=VN6NIXXOT6KUQE3JU6X35TPYNCI5K4KOSBLVXMOKSP6N2SYRFM6Q, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.486560086 (203h25m5.917640217s) [noise=true , address=104.36.82.2:4748 , piece_id=T6MOSFTUKYGVT2YSCEGXMSDXLULB6NI4RGGGB3SSARO7ALNZJN5Q, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.498376395 (203h25m5.929456526s) [noise=true , address=104.36.82.44:4896 , piece_id=2DQKKZFJX4N6W5UOOYX4QWSZ3GAFBMQIIDCKU3N3N5MLM4Q7T2AQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.512939881 (203h25m5.944020012s) [noise=true , address=45.91.116.12:4228 , piece_id=X5VGNN2WVGEW3E4MPUO5MQWZ7L2JMB345IQUUTBI4DYW5HESK33A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.512969688 (203h25m5.944049819s) [seg_pos={0 0}, upload=211594]: Segment reached optimal threshold of 14 pieces. Aug 17 04:51:11.513156405 (203h25m5.944236536s) [noise=true , address=77.90.31.13:4658 , piece_id=XONXAMV3VAHHSHNIDMAIMMD6KOCKLYJQAUTNVMSUXB6ZCWP5THDA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129aongkQE9HMNz8vsNNA5SvKbhsHqdt9UrtSTPJJpP9mkFsboX): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513220778 (203h25m5.944300909s) [noise=true , address=104.36.82.58:4295 , piece_id=3LNVXIW5IU2BQZRH6QAVV7A2PJ3L3YSGNOXBMIYOJMFBVHJZUPYA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oLa34qwPBAx3CTvYYj7GN3oxxWGJ9dAYXfke1eUftZc6xzfqg): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513269471 (203h25m5.944349603s) [noise=true , address=104.36.82.59:4331 , piece_id=SPNJSZRY7LNIS7ZCHJLOOWKVZU7AP3CODFM73CGOLQZU7HUCT2IQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513316901 (203h25m5.944397033s) [noise=true , address=104.36.82.18:4396 , piece_id=3CORBTMNXLB5AG6ZGAPLPBGGFEQ4SSMJHRGXPESWTJD5YPYVUKFA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1k5gzJBGWhpARcfiynburgfTqJ2t7sVNs1mAkUfToMG5saxPqx): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513333754 (203h25m5.944413885s) [seg_pos={0 0}, upload=211594]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 04:51:11.513406654 (203h25m5.944486775s) [upload=211594]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 04:51:11.513424287 (203h25m5.944504418s) [upload=211594]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:51:11.809871930 (203h25m6.240952061s) [upload=211594]: Done uploading segments... Aug 17 04:51:11.809892119 (203h25m6.24097225s) [upload=211594]: Upload finished. err: <nil> Duration: 5.697470422s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.................................X.....X....................] duration: 1.825613073s FAIL (2025-08-17 04:54:19 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 17 04:41:26.040732864 (203h15m20.471812985s) [upload=211420]: Starting upload Aug 17 04:41:26.040780384 (203h15m20.471860515s) [upload=211420]: Done starting upload Aug 17 04:41:26.040828978 (203h15m20.471909109s) [upload=211420]: Uploading segments... Aug 17 04:41:26.040863473 (203h15m20.471943604s) [upload=211420]: Got next segment. Inline: true Aug 17 04:41:26.040867290 (203h15m20.471947422s) [upload=211420]: Waiting for error group managing segments... Aug 17 04:41:26.040897880 (203h15m20.471978011s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:41:26.040901597 (203h15m20.471981728s) [upload=211420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:41:26.040904041 (203h15m20.471984162s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:41:26.462140724 (203h15m20.893220855s) [upload=211420]: Done uploading segments... Aug 17 04:41:26.462159310 (203h15m20.893239431s) [upload=211420]: Upload finished. err: <nil> Duration:* 1.825639204s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...................XX.......................................] duration: 6.459478286s FAIL (2025-08-17 04:54:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.459499707s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............XXXX...........................................] duration: 6.00664141s FAIL (2025-08-17 04:53:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.006669184s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 10 out of 60 is failed (threshold=3) [.......X..........XXXX.X.X.......X...X.................X....] duration: 5.764667719s FAIL (2025-08-17 04:53:52 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 04:51:09.056938814 (203h25m3.488018935s) [upload=211594]: Starting upload Aug 17 04:51:09.057017635 (203h25m3.488097766s) [upload=211594]: Done starting upload Aug 17 04:51:09.057190195 (203h25m3.488270326s) [upload=211594]: Uploading segments... Aug 17 04:51:09.057308371 (203h25m3.488388502s) [upload=211594]: Got next segment. Inline: false Aug 17 04:51:09.057342866 (203h25m3.488422997s) [upload=211594]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:51:09.057347857 (203h25m3.488427978s) [upload=211594]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:51:09.504198039 (203h25m3.93527816s) [seg_pos={0 0}, upload=211594]: Begin upload segment... Aug 17 04:51:09.504246441 (203h25m3.935326572s) [seg_pos={0 0}, upload=211594]: Done begin upload segment. Aug 17 04:51:09.504264686 (203h25m3.935344817s) [noise=true , address=104.36.82.2:4748 , piece_id=T6MOSFTUKYGVT2YSCEGXMSDXLULB6NI4RGGGB3SSARO7ALNZJN5Q, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504308099 (203h25m3.93538823s) [noise=true , address=104.36.82.19:4436 , piece_id=VN6NIXXOT6KUQE3JU6X35TPYNCI5K4KOSBLVXMOKSP6N2SYRFM6Q, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504588896 (203h25m3.935669017s) [noise=true , address=104.36.82.58:4295 , piece_id=3LNVXIW5IU2BQZRH6QAVV7A2PJ3L3YSGNOXBMIYOJMFBVHJZUPYA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504650284 (203h25m3.935730405s) [noise=true , address=64.49.13.169:4706 , piece_id=Y4SS3PCBRVCQS5N5N34Y6PGUJ5FPWOCHHO5KMU2PJ5CUXWTHW7RQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504844445 (203h25m3.935924566s) [noise=true , address=45.91.116.12:4228 , piece_id=X5VGNN2WVGEW3E4MPUO5MQWZ7L2JMB345IQUUTBI4DYW5HESK33A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504957571 (203h25m3.936037702s) [noise=true , address=104.36.82.23:4475 , piece_id=TYHG2VUDQ4RRMFLUOLV3U73G7JXA6ZRC757TD34Q2MZTJNRMUIZA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505108992 (203h25m3.936189123s) [noise=true , address=104.36.82.59:4331 , piece_id=SPNJSZRY7LNIS7ZCHJLOOWKVZU7AP3CODFM73CGOLQZU7HUCT2IQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505263607 (203h25m3.936343738s) [noise=true , address=66.23.193.175:4553 , piece_id=BXVNQMDXCQPEUSLIZ24VI22F7FOOJN2MEGJU6UI2SDQIZJSFQJ2A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505341626 (203h25m3.936421757s) [noise=true , address=77.90.31.10:4523 , piece_id=22C56K4AOQCL5N6WR7QECVAAXHOS3OUNMGWCRFHP5GKSTHIT7OLA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505565765 (203h25m3.936645896s) [noise=true , address=104.36.82.4:4836 , piece_id=JJWCMFYVQUU6WGEVBEJS4JPZQNNZERI5BVTBBAQQUZV3WD63NGIA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505597326 (203h25m3.936677457s) [noise=true , address=77.90.31.11:4563 , piece_id=QDC72JPWTU4LH5URAU44ECPV72L4NQPWGDDOHHVYSGNJTFS7C6XQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505853546 (203h25m3.936933677s) [noise=true , address=104.36.82.18:4396 , piece_id=3CORBTMNXLB5AG6ZGAPLPBGGFEQ4SSMJHRGXPESWTJD5YPYVUKFA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505894053 (203h25m3.936974184s) [noise=true , address=104.36.82.44:4896 , piece_id=2DQKKZFJX4N6W5UOOYX4QWSZ3GAFBMQIIDCKU3N3N5MLM4Q7T2AQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506090339 (203h25m3.937170471s) [noise=true , address=77.90.31.13:4658 , piece_id=XONXAMV3VAHHSHNIDMAIMMD6KOCKLYJQAUTNVMSUXB6ZCWP5THDA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506165653 (203h25m3.937245784s) [noise=true , address=104.36.82.5:4850 , piece_id=6ZYI2JS5HMQKTUM2XR4ZTSJMC3MHVILV4YOQ7ANA5PX3FDJXJ2DA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506275653 (203h25m3.937355774s) [noise=true , address=45.91.116.14:4199 , piece_id=HOTL5QUTJLBBZNF6A4WPRKMQXB72I3TUVSDYUVWMU7G4ZIXOIK3A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506396365 (203h25m3.937476496s) [noise=true , address=104.36.82.34:4565 , piece_id=LUXKT7N4OUT4UJA345KTE2T3QUD3LEGLUQKQTDZADYBOGE2Z5XNQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506467330 (203h25m3.937547461s) [noise=true , address=104.36.82.60:4354 , piece_id=M3INHRZ2XHGLNFSI7OHKHKNCVTSOWTSCJJCGHTTYCAJGFLDWLL7A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:10.617749206 (203h25m5.048829337s) [upload=211594]: Next returned nil segment Aug 17 04:51:10.617765637 (203h25m5.048845758s) [upload=211594]: Waiting for error group managing segments... Aug 17 04:51:10.899081476 (203h25m5.330161607s) [noise=true , address=64.49.13.169:4706 , piece_id=Y4SS3PCBRVCQS5N5N34Y6PGUJ5FPWOCHHO5KMU2PJ5CUXWTHW7RQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:10.959773754 (203h25m5.390853875s) [noise=true , address=66.23.193.175:4553 , piece_id=BXVNQMDXCQPEUSLIZ24VI22F7FOOJN2MEGJU6UI2SDQIZJSFQJ2A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.387426163 (203h25m5.818506294s) [noise=true , address=77.90.31.11:4563 , piece_id=QDC72JPWTU4LH5URAU44ECPV72L4NQPWGDDOHHVYSGNJTFS7C6XQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.392712709 (203h25m5.82379284s) [noise=true , address=45.91.116.14:4199 , piece_id=HOTL5QUTJLBBZNF6A4WPRKMQXB72I3TUVSDYUVWMU7G4ZIXOIK3A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.399286516 (203h25m5.830366637s) [noise=true , address=104.36.82.4:4836 , piece_id=JJWCMFYVQUU6WGEVBEJS4JPZQNNZERI5BVTBBAQQUZV3WD63NGIA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.412750859 (203h25m5.84383099s) [noise=true , address=104.36.82.23:4475 , piece_id=TYHG2VUDQ4RRMFLUOLV3U73G7JXA6ZRC757TD34Q2MZTJNRMUIZA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.439625902 (203h25m5.870706033s) [noise=true , address=77.90.31.10:4523 , piece_id=22C56K4AOQCL5N6WR7QECVAAXHOS3OUNMGWCRFHP5GKSTHIT7OLA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.465577498 (203h25m5.896657629s) [noise=true , address=104.36.82.34:4565 , piece_id=LUXKT7N4OUT4UJA345KTE2T3QUD3LEGLUQKQTDZADYBOGE2Z5XNQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.469847087 (203h25m5.900927218s) [noise=true , address=104.36.82.60:4354 , piece_id=M3INHRZ2XHGLNFSI7OHKHKNCVTSOWTSCJJCGHTTYCAJGFLDWLL7A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.470697434 (203h25m5.901777555s) [noise=true , address=104.36.82.5:4850 , piece_id=6ZYI2JS5HMQKTUM2XR4ZTSJMC3MHVILV4YOQ7ANA5PX3FDJXJ2DA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.470949846 (203h25m5.902029978s) [noise=true , address=104.36.82.19:4436 , piece_id=VN6NIXXOT6KUQE3JU6X35TPYNCI5K4KOSBLVXMOKSP6N2SYRFM6Q, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.486560086 (203h25m5.917640217s) [noise=true , address=104.36.82.2:4748 , piece_id=T6MOSFTUKYGVT2YSCEGXMSDXLULB6NI4RGGGB3SSARO7ALNZJN5Q, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.498376395 (203h25m5.929456526s) [noise=true , address=104.36.82.44:4896 , piece_id=2DQKKZFJX4N6W5UOOYX4QWSZ3GAFBMQIIDCKU3N3N5MLM4Q7T2AQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.512939881 (203h25m5.944020012s) [noise=true , address=45.91.116.12:4228 , piece_id=X5VGNN2WVGEW3E4MPUO5MQWZ7L2JMB345IQUUTBI4DYW5HESK33A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.512969688 (203h25m5.944049819s) [seg_pos={0 0}, upload=211594]: Segment reached optimal threshold of 14 pieces. Aug 17 04:51:11.513156405 (203h25m5.944236536s) [noise=true , address=77.90.31.13:4658 , piece_id=XONXAMV3VAHHSHNIDMAIMMD6KOCKLYJQAUTNVMSUXB6ZCWP5THDA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129aongkQE9HMNz8vsNNA5SvKbhsHqdt9UrtSTPJJpP9mkFsboX): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513220778 (203h25m5.944300909s) [noise=true , address=104.36.82.58:4295 , piece_id=3LNVXIW5IU2BQZRH6QAVV7A2PJ3L3YSGNOXBMIYOJMFBVHJZUPYA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oLa34qwPBAx3CTvYYj7GN3oxxWGJ9dAYXfke1eUftZc6xzfqg): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513269471 (203h25m5.944349603s) [noise=true , address=104.36.82.59:4331 , piece_id=SPNJSZRY7LNIS7ZCHJLOOWKVZU7AP3CODFM73CGOLQZU7HUCT2IQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513316901 (203h25m5.944397033s) [noise=true , address=104.36.82.18:4396 , piece_id=3CORBTMNXLB5AG6ZGAPLPBGGFEQ4SSMJHRGXPESWTJD5YPYVUKFA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1k5gzJBGWhpARcfiynburgfTqJ2t7sVNs1mAkUfToMG5saxPqx): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513333754 (203h25m5.944413885s) [seg_pos={0 0}, upload=211594]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 04:51:11.513406654 (203h25m5.944486775s) [upload=211594]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 04:51:11.513424287 (203h25m5.944504418s) [upload=211594]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:51:11.809871930 (203h25m6.240952061s) [upload=211594]: Done uploading segments... Aug 17 04:51:11.809892119 (203h25m6.24097225s) [upload=211594]: Upload finished. err: <nil> Duration: 5.764715982s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............XXXX..........................................] duration: 5.927336721s FAIL (2025-08-17 04:53:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 04:23:03.873697274 (202h56m58.304777405s) [upload=211112]: Starting upload Aug 17 04:23:03.873756368 (202h56m58.304836499s) [upload=211112]: Done starting upload Aug 17 04:23:03.873839848 (202h56m58.304919989s) [upload=211112]: Uploading segments... Aug 17 04:23:03.873950850 (202h56m58.305030981s) [upload=211112]: Got next segment. Inline: false Aug 17 04:23:03.873976509 (202h56m58.30505664s) [upload=211112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:03.873980757 (202h56m58.305060888s) [upload=211112]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:04.113727713 (202h56m58.544807844s) [seg_pos={0 0}, upload=211112]: Begin upload segment... Aug 17 04:23:04.113774051 (202h56m58.544854182s) [seg_pos={0 0}, upload=211112]: Done begin upload segment. Aug 17 04:23:04.113820890 (202h56m58.544901021s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.113845989 (202h56m58.54492612s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114195037 (202h56m58.545275168s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114281412 (202h56m58.545361543s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114482115 (202h56m58.545562237s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114570385 (202h56m58.545650516s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114757392 (202h56m58.545837523s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114874727 (202h56m58.545954858s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115030715 (202h56m58.546110846s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115194288 (202h56m58.546274419s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115307967 (202h56m58.546388098s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115535692 (202h56m58.546615823s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115754130 (202h56m58.546834261s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116079182 (202h56m58.547159323s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116350181 (202h56m58.547430312s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116554471 (202h56m58.547634602s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:05.868765063 (202h57m0.299845195s) [upload=211112]: Next returned nil segment Aug 17 04:23:05.868801043 (202h57m0.299881164s) [upload=211112]: Waiting for error group managing segments... Aug 17 04:23:06.147430529 (202h57m0.57851066s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.182400314 (202h57m0.613480445s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.191910532 (202h57m0.622990663s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.197391631 (202h57m0.628471782s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.198069487 (202h57m0.629149618s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.200597014 (202h57m0.631677145s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.288053251 (202h57m0.719133372s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.298810435 (202h57m0.729890566s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.301049520 (202h57m0.732129651s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.402776985 (202h57m0.833857116s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:33.223003412 (202h57m27.654083543s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: write tcp 188.166.225.170:42258->104.36.82.5:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223050702 (202h57m27.654130823s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:52308->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223090619 (202h57m27.65417075s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 188.166.225.170:53078->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223104976 (202h57m27.654185097s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:56176->104.36.82.4:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223149581 (202h57m27.654229712s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:40582->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223157536 (202h57m27.654237667s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 188.166.225.170:32954->104.36.82.9:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223177415 (202h57m27.654257536s) [seg_pos={0 0}, upload=211112]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 04:23:33.448862924 (202h57m27.879943045s) [upload=211112]: Done uploading segments... Aug 17 04:23:33.448886830 (202h57m27.879966961s) [upload=211112]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.927363232s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............XXX...........................................] duration: 5.659674916s FAIL (2025-08-17 04:53:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== g_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761278730 (202h56m21.192358861s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761661883 (202h56m21.192742014s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761716548 (202h56m21.192796679s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761972016 (202h56m21.193052147s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762254878 (202h56m21.193335009s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762523682 (202h56m21.193603803s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762864294 (202h56m21.193944415s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763179537 (202h56m21.194259668s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763478499 (202h56m21.19455864s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763757683 (202h56m21.194837814s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763988195 (202h56m21.195068326s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764273320 (202h56m21.195353451s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764555100 (202h56m21.195635221s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764757416 (202h56m21.195837547s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.765868993 (202h56m21.196949124s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.767152809 (202h56m21.19823294s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.768577886 (202h56m21.199658017s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:27.934309011 (202h56m22.365389142s) [upload=211101]: Next returned nil segment Aug 17 04:22:27.934342545 (202h56m22.365422666s) [upload=211101]: Waiting for error group managing segments... Aug 17 04:22:28.231985083 (202h56m22.663065214s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.670701256 (202h56m23.101781427s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.671952431 (202h56m23.103032562s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.698468278 (202h56m23.129548409s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.720171187 (202h56m23.151251318s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.731591429 (202h56m23.16267156s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.798148986 (202h56m23.229229117s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.804849037 (202h56m23.235929168s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.833066822 (202h56m23.264147013s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.846684710 (202h56m23.277764831s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:23:25.582810947 (202h57m20.013891078s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12TU4zvu38mF1fNSergCtMj2cK9eu57Rdfpt5MVk3tc4MZF6dNr, address:104.36.82.34:4558): protocol: rpc: dial tcp 104.36.82.34:4558: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582856475 (202h57m20.013936606s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12D9nh4XFmsjjFcZEMjaL4uw4Whsa8nfMrAsYPu6NVpsxgb3jvf, address:104.36.82.35:4594): protocol: rpc: dial tcp 104.36.82.35:4594: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582973238 (202h57m20.014053369s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1WvyEw8XevCiQ8yRgzyMo7WTVzpEJ158ErKfAeV4dSTBkS1oRF, address:104.36.82.19:4436): protocol: rpc: dial tcp 104.36.82.19:4436: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583033393 (202h57m20.014113524s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:104.36.82.58:4283): protocol: rpc: dial tcp 104.36.82.58:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583069241 (202h57m20.014149362s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:126Z6tvp49dLLdj3T5snJc4tBVGzcX5UBaGafFN3PkhF92vxpfs, address:104.36.82.50:4047): protocol: rpc: dial tcp 104.36.82.50:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583076716 (202h57m20.014156837s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1zzW2z6noGQT6xdSmwSQBNYwip6gZjnmqZg7qdb6BP3nHyCLFz, address:104.36.82.39:4753): protocol: rpc: dial tcp 104.36.82.39:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583107535 (202h57m20.014187666s) [noise=true , address=104.36.82.23:4474 , piece_id=RITPHITVW655UQETT6UKS7BZO2S3F4P2MJIO4REOUKYYP4E2CDVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 188.166.225.170:36642->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583165355 (202h57m20.014245487s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh, address:104.36.82.5:4855): protocol: write tcp 188.166.225.170:47052->104.36.82.5:4855: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583194000 (202h57m20.014274131s) [seg_pos={0 0}, upload=211101]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:25.810738626 (202h57m20.241818757s) [upload=211101]: Done uploading segments... Aug 17 04:23:25.810761991 (202h57m20.241842122s) [upload=211101]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.659703541s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..................................X.....X...................] duration: 1.849060186s FAIL (2025-08-17 04:53:28 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 17 04:41:26.040732864 (203h15m20.471812985s) [upload=211420]: Starting upload Aug 17 04:41:26.040780384 (203h15m20.471860515s) [upload=211420]: Done starting upload Aug 17 04:41:26.040828978 (203h15m20.471909109s) [upload=211420]: Uploading segments... Aug 17 04:41:26.040863473 (203h15m20.471943604s) [upload=211420]: Got next segment. Inline: true Aug 17 04:41:26.040867290 (203h15m20.471947422s) [upload=211420]: Waiting for error group managing segments... Aug 17 04:41:26.040897880 (203h15m20.471978011s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:41:26.040901597 (203h15m20.471981728s) [upload=211420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:41:26.040904041 (203h15m20.471984162s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:41:26.462140724 (203h15m20.893220855s) [upload=211420]: Done uploading segments... Aug 17 04:41:26.462159310 (203h15m20.893239431s) [upload=211420]: Upload finished. err: <nil> Duration:* 1.849087889s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....................XX......................................] duration: 6.751425005s FAIL (2025-08-17 04:53:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.751470762s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............XXXX..........................................] duration: 6.264973949s FAIL (2025-08-17 04:53:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.264994649s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............XXX..........................................] duration: 5.754936183s FAIL (2025-08-17 04:53:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== g_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761278730 (202h56m21.192358861s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761661883 (202h56m21.192742014s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761716548 (202h56m21.192796679s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761972016 (202h56m21.193052147s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762254878 (202h56m21.193335009s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762523682 (202h56m21.193603803s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762864294 (202h56m21.193944415s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763179537 (202h56m21.194259668s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763478499 (202h56m21.19455864s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763757683 (202h56m21.194837814s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763988195 (202h56m21.195068326s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764273320 (202h56m21.195353451s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764555100 (202h56m21.195635221s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764757416 (202h56m21.195837547s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.765868993 (202h56m21.196949124s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.767152809 (202h56m21.19823294s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.768577886 (202h56m21.199658017s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:27.934309011 (202h56m22.365389142s) [upload=211101]: Next returned nil segment Aug 17 04:22:27.934342545 (202h56m22.365422666s) [upload=211101]: Waiting for error group managing segments... Aug 17 04:22:28.231985083 (202h56m22.663065214s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.670701256 (202h56m23.101781427s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.671952431 (202h56m23.103032562s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.698468278 (202h56m23.129548409s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.720171187 (202h56m23.151251318s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.731591429 (202h56m23.16267156s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.798148986 (202h56m23.229229117s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.804849037 (202h56m23.235929168s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.833066822 (202h56m23.264147013s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.846684710 (202h56m23.277764831s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:23:25.582810947 (202h57m20.013891078s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12TU4zvu38mF1fNSergCtMj2cK9eu57Rdfpt5MVk3tc4MZF6dNr, address:104.36.82.34:4558): protocol: rpc: dial tcp 104.36.82.34:4558: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582856475 (202h57m20.013936606s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12D9nh4XFmsjjFcZEMjaL4uw4Whsa8nfMrAsYPu6NVpsxgb3jvf, address:104.36.82.35:4594): protocol: rpc: dial tcp 104.36.82.35:4594: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582973238 (202h57m20.014053369s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1WvyEw8XevCiQ8yRgzyMo7WTVzpEJ158ErKfAeV4dSTBkS1oRF, address:104.36.82.19:4436): protocol: rpc: dial tcp 104.36.82.19:4436: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583033393 (202h57m20.014113524s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:104.36.82.58:4283): protocol: rpc: dial tcp 104.36.82.58:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583069241 (202h57m20.014149362s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:126Z6tvp49dLLdj3T5snJc4tBVGzcX5UBaGafFN3PkhF92vxpfs, address:104.36.82.50:4047): protocol: rpc: dial tcp 104.36.82.50:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583076716 (202h57m20.014156837s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1zzW2z6noGQT6xdSmwSQBNYwip6gZjnmqZg7qdb6BP3nHyCLFz, address:104.36.82.39:4753): protocol: rpc: dial tcp 104.36.82.39:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583107535 (202h57m20.014187666s) [noise=true , address=104.36.82.23:4474 , piece_id=RITPHITVW655UQETT6UKS7BZO2S3F4P2MJIO4REOUKYYP4E2CDVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 188.166.225.170:36642->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583165355 (202h57m20.014245487s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh, address:104.36.82.5:4855): protocol: write tcp 188.166.225.170:47052->104.36.82.5:4855: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583194000 (202h57m20.014274131s) [seg_pos={0 0}, upload=211101]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:25.810738626 (202h57m20.241818757s) [upload=211101]: Done uploading segments... Aug 17 04:23:25.810761991 (202h57m20.241842122s) [upload=211101]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.754965189s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...................................X.....X..................] duration: 1.824767043s FAIL (2025-08-17 04:52:50 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 17 04:41:26.040732864 (203h15m20.471812985s) [upload=211420]: Starting upload Aug 17 04:41:26.040780384 (203h15m20.471860515s) [upload=211420]: Done starting upload Aug 17 04:41:26.040828978 (203h15m20.471909109s) [upload=211420]: Uploading segments... Aug 17 04:41:26.040863473 (203h15m20.471943604s) [upload=211420]: Got next segment. Inline: true Aug 17 04:41:26.040867290 (203h15m20.471947422s) [upload=211420]: Waiting for error group managing segments... Aug 17 04:41:26.040897880 (203h15m20.471978011s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:41:26.040901597 (203h15m20.471981728s) [upload=211420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:41:26.040904041 (203h15m20.471984162s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:41:26.462140724 (203h15m20.893220855s) [upload=211420]: Done uploading segments... Aug 17 04:41:26.462159310 (203h15m20.893239431s) [upload=211420]: Upload finished. err: <nil> Duration:* 1.824796649s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 10 out of 60 is failed (threshold=3) [........X..........XXXX.X.X.......X...X.................X...] duration: 5.501117974s FAIL (2025-08-17 04:52:43 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 04:51:09.056938814 (203h25m3.488018935s) [upload=211594]: Starting upload Aug 17 04:51:09.057017635 (203h25m3.488097766s) [upload=211594]: Done starting upload Aug 17 04:51:09.057190195 (203h25m3.488270326s) [upload=211594]: Uploading segments... Aug 17 04:51:09.057308371 (203h25m3.488388502s) [upload=211594]: Got next segment. Inline: false Aug 17 04:51:09.057342866 (203h25m3.488422997s) [upload=211594]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:51:09.057347857 (203h25m3.488427978s) [upload=211594]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:51:09.504198039 (203h25m3.93527816s) [seg_pos={0 0}, upload=211594]: Begin upload segment... Aug 17 04:51:09.504246441 (203h25m3.935326572s) [seg_pos={0 0}, upload=211594]: Done begin upload segment. Aug 17 04:51:09.504264686 (203h25m3.935344817s) [noise=true , address=104.36.82.2:4748 , piece_id=T6MOSFTUKYGVT2YSCEGXMSDXLULB6NI4RGGGB3SSARO7ALNZJN5Q, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504308099 (203h25m3.93538823s) [noise=true , address=104.36.82.19:4436 , piece_id=VN6NIXXOT6KUQE3JU6X35TPYNCI5K4KOSBLVXMOKSP6N2SYRFM6Q, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504588896 (203h25m3.935669017s) [noise=true , address=104.36.82.58:4295 , piece_id=3LNVXIW5IU2BQZRH6QAVV7A2PJ3L3YSGNOXBMIYOJMFBVHJZUPYA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504650284 (203h25m3.935730405s) [noise=true , address=64.49.13.169:4706 , piece_id=Y4SS3PCBRVCQS5N5N34Y6PGUJ5FPWOCHHO5KMU2PJ5CUXWTHW7RQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504844445 (203h25m3.935924566s) [noise=true , address=45.91.116.12:4228 , piece_id=X5VGNN2WVGEW3E4MPUO5MQWZ7L2JMB345IQUUTBI4DYW5HESK33A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504957571 (203h25m3.936037702s) [noise=true , address=104.36.82.23:4475 , piece_id=TYHG2VUDQ4RRMFLUOLV3U73G7JXA6ZRC757TD34Q2MZTJNRMUIZA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505108992 (203h25m3.936189123s) [noise=true , address=104.36.82.59:4331 , piece_id=SPNJSZRY7LNIS7ZCHJLOOWKVZU7AP3CODFM73CGOLQZU7HUCT2IQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505263607 (203h25m3.936343738s) [noise=true , address=66.23.193.175:4553 , piece_id=BXVNQMDXCQPEUSLIZ24VI22F7FOOJN2MEGJU6UI2SDQIZJSFQJ2A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505341626 (203h25m3.936421757s) [noise=true , address=77.90.31.10:4523 , piece_id=22C56K4AOQCL5N6WR7QECVAAXHOS3OUNMGWCRFHP5GKSTHIT7OLA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505565765 (203h25m3.936645896s) [noise=true , address=104.36.82.4:4836 , piece_id=JJWCMFYVQUU6WGEVBEJS4JPZQNNZERI5BVTBBAQQUZV3WD63NGIA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505597326 (203h25m3.936677457s) [noise=true , address=77.90.31.11:4563 , piece_id=QDC72JPWTU4LH5URAU44ECPV72L4NQPWGDDOHHVYSGNJTFS7C6XQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505853546 (203h25m3.936933677s) [noise=true , address=104.36.82.18:4396 , piece_id=3CORBTMNXLB5AG6ZGAPLPBGGFEQ4SSMJHRGXPESWTJD5YPYVUKFA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505894053 (203h25m3.936974184s) [noise=true , address=104.36.82.44:4896 , piece_id=2DQKKZFJX4N6W5UOOYX4QWSZ3GAFBMQIIDCKU3N3N5MLM4Q7T2AQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506090339 (203h25m3.937170471s) [noise=true , address=77.90.31.13:4658 , piece_id=XONXAMV3VAHHSHNIDMAIMMD6KOCKLYJQAUTNVMSUXB6ZCWP5THDA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506165653 (203h25m3.937245784s) [noise=true , address=104.36.82.5:4850 , piece_id=6ZYI2JS5HMQKTUM2XR4ZTSJMC3MHVILV4YOQ7ANA5PX3FDJXJ2DA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506275653 (203h25m3.937355774s) [noise=true , address=45.91.116.14:4199 , piece_id=HOTL5QUTJLBBZNF6A4WPRKMQXB72I3TUVSDYUVWMU7G4ZIXOIK3A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506396365 (203h25m3.937476496s) [noise=true , address=104.36.82.34:4565 , piece_id=LUXKT7N4OUT4UJA345KTE2T3QUD3LEGLUQKQTDZADYBOGE2Z5XNQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506467330 (203h25m3.937547461s) [noise=true , address=104.36.82.60:4354 , piece_id=M3INHRZ2XHGLNFSI7OHKHKNCVTSOWTSCJJCGHTTYCAJGFLDWLL7A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:10.617749206 (203h25m5.048829337s) [upload=211594]: Next returned nil segment Aug 17 04:51:10.617765637 (203h25m5.048845758s) [upload=211594]: Waiting for error group managing segments... Aug 17 04:51:10.899081476 (203h25m5.330161607s) [noise=true , address=64.49.13.169:4706 , piece_id=Y4SS3PCBRVCQS5N5N34Y6PGUJ5FPWOCHHO5KMU2PJ5CUXWTHW7RQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:10.959773754 (203h25m5.390853875s) [noise=true , address=66.23.193.175:4553 , piece_id=BXVNQMDXCQPEUSLIZ24VI22F7FOOJN2MEGJU6UI2SDQIZJSFQJ2A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.387426163 (203h25m5.818506294s) [noise=true , address=77.90.31.11:4563 , piece_id=QDC72JPWTU4LH5URAU44ECPV72L4NQPWGDDOHHVYSGNJTFS7C6XQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.392712709 (203h25m5.82379284s) [noise=true , address=45.91.116.14:4199 , piece_id=HOTL5QUTJLBBZNF6A4WPRKMQXB72I3TUVSDYUVWMU7G4ZIXOIK3A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.399286516 (203h25m5.830366637s) [noise=true , address=104.36.82.4:4836 , piece_id=JJWCMFYVQUU6WGEVBEJS4JPZQNNZERI5BVTBBAQQUZV3WD63NGIA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.412750859 (203h25m5.84383099s) [noise=true , address=104.36.82.23:4475 , piece_id=TYHG2VUDQ4RRMFLUOLV3U73G7JXA6ZRC757TD34Q2MZTJNRMUIZA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.439625902 (203h25m5.870706033s) [noise=true , address=77.90.31.10:4523 , piece_id=22C56K4AOQCL5N6WR7QECVAAXHOS3OUNMGWCRFHP5GKSTHIT7OLA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.465577498 (203h25m5.896657629s) [noise=true , address=104.36.82.34:4565 , piece_id=LUXKT7N4OUT4UJA345KTE2T3QUD3LEGLUQKQTDZADYBOGE2Z5XNQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.469847087 (203h25m5.900927218s) [noise=true , address=104.36.82.60:4354 , piece_id=M3INHRZ2XHGLNFSI7OHKHKNCVTSOWTSCJJCGHTTYCAJGFLDWLL7A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.470697434 (203h25m5.901777555s) [noise=true , address=104.36.82.5:4850 , piece_id=6ZYI2JS5HMQKTUM2XR4ZTSJMC3MHVILV4YOQ7ANA5PX3FDJXJ2DA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.470949846 (203h25m5.902029978s) [noise=true , address=104.36.82.19:4436 , piece_id=VN6NIXXOT6KUQE3JU6X35TPYNCI5K4KOSBLVXMOKSP6N2SYRFM6Q, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.486560086 (203h25m5.917640217s) [noise=true , address=104.36.82.2:4748 , piece_id=T6MOSFTUKYGVT2YSCEGXMSDXLULB6NI4RGGGB3SSARO7ALNZJN5Q, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.498376395 (203h25m5.929456526s) [noise=true , address=104.36.82.44:4896 , piece_id=2DQKKZFJX4N6W5UOOYX4QWSZ3GAFBMQIIDCKU3N3N5MLM4Q7T2AQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.512939881 (203h25m5.944020012s) [noise=true , address=45.91.116.12:4228 , piece_id=X5VGNN2WVGEW3E4MPUO5MQWZ7L2JMB345IQUUTBI4DYW5HESK33A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.512969688 (203h25m5.944049819s) [seg_pos={0 0}, upload=211594]: Segment reached optimal threshold of 14 pieces. Aug 17 04:51:11.513156405 (203h25m5.944236536s) [noise=true , address=77.90.31.13:4658 , piece_id=XONXAMV3VAHHSHNIDMAIMMD6KOCKLYJQAUTNVMSUXB6ZCWP5THDA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129aongkQE9HMNz8vsNNA5SvKbhsHqdt9UrtSTPJJpP9mkFsboX): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513220778 (203h25m5.944300909s) [noise=true , address=104.36.82.58:4295 , piece_id=3LNVXIW5IU2BQZRH6QAVV7A2PJ3L3YSGNOXBMIYOJMFBVHJZUPYA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oLa34qwPBAx3CTvYYj7GN3oxxWGJ9dAYXfke1eUftZc6xzfqg): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513269471 (203h25m5.944349603s) [noise=true , address=104.36.82.59:4331 , piece_id=SPNJSZRY7LNIS7ZCHJLOOWKVZU7AP3CODFM73CGOLQZU7HUCT2IQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513316901 (203h25m5.944397033s) [noise=true , address=104.36.82.18:4396 , piece_id=3CORBTMNXLB5AG6ZGAPLPBGGFEQ4SSMJHRGXPESWTJD5YPYVUKFA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1k5gzJBGWhpARcfiynburgfTqJ2t7sVNs1mAkUfToMG5saxPqx): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513333754 (203h25m5.944413885s) [seg_pos={0 0}, upload=211594]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 04:51:11.513406654 (203h25m5.944486775s) [upload=211594]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 04:51:11.513424287 (203h25m5.944504418s) [upload=211594]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:51:11.809871930 (203h25m6.240952061s) [upload=211594]: Done uploading segments... Aug 17 04:51:11.809892119 (203h25m6.24097225s) [upload=211594]: Upload finished. err: <nil> Duration: 5.501145435s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............XXXX.........................................] duration: 5.810966977s FAIL (2025-08-17 04:52:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 04:23:03.873697274 (202h56m58.304777405s) [upload=211112]: Starting upload Aug 17 04:23:03.873756368 (202h56m58.304836499s) [upload=211112]: Done starting upload Aug 17 04:23:03.873839848 (202h56m58.304919989s) [upload=211112]: Uploading segments... Aug 17 04:23:03.873950850 (202h56m58.305030981s) [upload=211112]: Got next segment. Inline: false Aug 17 04:23:03.873976509 (202h56m58.30505664s) [upload=211112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:03.873980757 (202h56m58.305060888s) [upload=211112]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:04.113727713 (202h56m58.544807844s) [seg_pos={0 0}, upload=211112]: Begin upload segment... Aug 17 04:23:04.113774051 (202h56m58.544854182s) [seg_pos={0 0}, upload=211112]: Done begin upload segment. Aug 17 04:23:04.113820890 (202h56m58.544901021s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.113845989 (202h56m58.54492612s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114195037 (202h56m58.545275168s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114281412 (202h56m58.545361543s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114482115 (202h56m58.545562237s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114570385 (202h56m58.545650516s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114757392 (202h56m58.545837523s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114874727 (202h56m58.545954858s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115030715 (202h56m58.546110846s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115194288 (202h56m58.546274419s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115307967 (202h56m58.546388098s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115535692 (202h56m58.546615823s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115754130 (202h56m58.546834261s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116079182 (202h56m58.547159323s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116350181 (202h56m58.547430312s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116554471 (202h56m58.547634602s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:05.868765063 (202h57m0.299845195s) [upload=211112]: Next returned nil segment Aug 17 04:23:05.868801043 (202h57m0.299881164s) [upload=211112]: Waiting for error group managing segments... Aug 17 04:23:06.147430529 (202h57m0.57851066s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.182400314 (202h57m0.613480445s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.191910532 (202h57m0.622990663s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.197391631 (202h57m0.628471782s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.198069487 (202h57m0.629149618s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.200597014 (202h57m0.631677145s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.288053251 (202h57m0.719133372s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.298810435 (202h57m0.729890566s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.301049520 (202h57m0.732129651s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.402776985 (202h57m0.833857116s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:33.223003412 (202h57m27.654083543s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: write tcp 188.166.225.170:42258->104.36.82.5:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223050702 (202h57m27.654130823s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:52308->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223090619 (202h57m27.65417075s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 188.166.225.170:53078->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223104976 (202h57m27.654185097s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:56176->104.36.82.4:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223149581 (202h57m27.654229712s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:40582->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223157536 (202h57m27.654237667s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 188.166.225.170:32954->104.36.82.9:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223177415 (202h57m27.654257536s) [seg_pos={0 0}, upload=211112]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 04:23:33.448862924 (202h57m27.879943045s) [upload=211112]: Done uploading segments... Aug 17 04:23:33.448886830 (202h57m27.879966961s) [upload=211112]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.811008506s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [....................................X.....X.................] duration: 1.839712112s FAIL (2025-08-17 04:52:29 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 17 04:41:26.040732864 (203h15m20.471812985s) [upload=211420]: Starting upload Aug 17 04:41:26.040780384 (203h15m20.471860515s) [upload=211420]: Done starting upload Aug 17 04:41:26.040828978 (203h15m20.471909109s) [upload=211420]: Uploading segments... Aug 17 04:41:26.040863473 (203h15m20.471943604s) [upload=211420]: Got next segment. Inline: true Aug 17 04:41:26.040867290 (203h15m20.471947422s) [upload=211420]: Waiting for error group managing segments... Aug 17 04:41:26.040897880 (203h15m20.471978011s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:41:26.040901597 (203h15m20.471981728s) [upload=211420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:41:26.040904041 (203h15m20.471984162s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:41:26.462140724 (203h15m20.893220855s) [upload=211420]: Done uploading segments... Aug 17 04:41:26.462159310 (203h15m20.893239431s) [upload=211420]: Upload finished. err: <nil> Duration:* 1.839739874s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [................XXX.........................................] duration: 6.025249706s FAIL (2025-08-17 04:52:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== g_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761278730 (202h56m21.192358861s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761661883 (202h56m21.192742014s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761716548 (202h56m21.192796679s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761972016 (202h56m21.193052147s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762254878 (202h56m21.193335009s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762523682 (202h56m21.193603803s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762864294 (202h56m21.193944415s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763179537 (202h56m21.194259668s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763478499 (202h56m21.19455864s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763757683 (202h56m21.194837814s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763988195 (202h56m21.195068326s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764273320 (202h56m21.195353451s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764555100 (202h56m21.195635221s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764757416 (202h56m21.195837547s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.765868993 (202h56m21.196949124s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.767152809 (202h56m21.19823294s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.768577886 (202h56m21.199658017s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:27.934309011 (202h56m22.365389142s) [upload=211101]: Next returned nil segment Aug 17 04:22:27.934342545 (202h56m22.365422666s) [upload=211101]: Waiting for error group managing segments... Aug 17 04:22:28.231985083 (202h56m22.663065214s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.670701256 (202h56m23.101781427s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.671952431 (202h56m23.103032562s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.698468278 (202h56m23.129548409s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.720171187 (202h56m23.151251318s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.731591429 (202h56m23.16267156s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.798148986 (202h56m23.229229117s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.804849037 (202h56m23.235929168s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.833066822 (202h56m23.264147013s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.846684710 (202h56m23.277764831s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:23:25.582810947 (202h57m20.013891078s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12TU4zvu38mF1fNSergCtMj2cK9eu57Rdfpt5MVk3tc4MZF6dNr, address:104.36.82.34:4558): protocol: rpc: dial tcp 104.36.82.34:4558: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582856475 (202h57m20.013936606s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12D9nh4XFmsjjFcZEMjaL4uw4Whsa8nfMrAsYPu6NVpsxgb3jvf, address:104.36.82.35:4594): protocol: rpc: dial tcp 104.36.82.35:4594: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582973238 (202h57m20.014053369s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1WvyEw8XevCiQ8yRgzyMo7WTVzpEJ158ErKfAeV4dSTBkS1oRF, address:104.36.82.19:4436): protocol: rpc: dial tcp 104.36.82.19:4436: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583033393 (202h57m20.014113524s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:104.36.82.58:4283): protocol: rpc: dial tcp 104.36.82.58:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583069241 (202h57m20.014149362s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:126Z6tvp49dLLdj3T5snJc4tBVGzcX5UBaGafFN3PkhF92vxpfs, address:104.36.82.50:4047): protocol: rpc: dial tcp 104.36.82.50:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583076716 (202h57m20.014156837s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1zzW2z6noGQT6xdSmwSQBNYwip6gZjnmqZg7qdb6BP3nHyCLFz, address:104.36.82.39:4753): protocol: rpc: dial tcp 104.36.82.39:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583107535 (202h57m20.014187666s) [noise=true , address=104.36.82.23:4474 , piece_id=RITPHITVW655UQETT6UKS7BZO2S3F4P2MJIO4REOUKYYP4E2CDVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 188.166.225.170:36642->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583165355 (202h57m20.014245487s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh, address:104.36.82.5:4855): protocol: write tcp 188.166.225.170:47052->104.36.82.5:4855: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583194000 (202h57m20.014274131s) [seg_pos={0 0}, upload=211101]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:25.810738626 (202h57m20.241818757s) [upload=211101]: Done uploading segments... Aug 17 04:23:25.810761991 (202h57m20.241842122s) [upload=211101]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.025400415s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.....................XX.....................................] duration: 6.452382078s FAIL (2025-08-17 04:52:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.452404001s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............XXXX.........................................] duration: 6.27038628s FAIL (2025-08-17 04:52:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.270411478s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 10 out of 60 is failed (threshold=3) [.........X..........XXXX.X.X.......X...X.................X..] duration: 5.567038385s FAIL (2025-08-17 04:52:11 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 04:51:09.056938814 (203h25m3.488018935s) [upload=211594]: Starting upload Aug 17 04:51:09.057017635 (203h25m3.488097766s) [upload=211594]: Done starting upload Aug 17 04:51:09.057190195 (203h25m3.488270326s) [upload=211594]: Uploading segments... Aug 17 04:51:09.057308371 (203h25m3.488388502s) [upload=211594]: Got next segment. Inline: false Aug 17 04:51:09.057342866 (203h25m3.488422997s) [upload=211594]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:51:09.057347857 (203h25m3.488427978s) [upload=211594]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:51:09.504198039 (203h25m3.93527816s) [seg_pos={0 0}, upload=211594]: Begin upload segment... Aug 17 04:51:09.504246441 (203h25m3.935326572s) [seg_pos={0 0}, upload=211594]: Done begin upload segment. Aug 17 04:51:09.504264686 (203h25m3.935344817s) [noise=true , address=104.36.82.2:4748 , piece_id=T6MOSFTUKYGVT2YSCEGXMSDXLULB6NI4RGGGB3SSARO7ALNZJN5Q, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504308099 (203h25m3.93538823s) [noise=true , address=104.36.82.19:4436 , piece_id=VN6NIXXOT6KUQE3JU6X35TPYNCI5K4KOSBLVXMOKSP6N2SYRFM6Q, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504588896 (203h25m3.935669017s) [noise=true , address=104.36.82.58:4295 , piece_id=3LNVXIW5IU2BQZRH6QAVV7A2PJ3L3YSGNOXBMIYOJMFBVHJZUPYA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504650284 (203h25m3.935730405s) [noise=true , address=64.49.13.169:4706 , piece_id=Y4SS3PCBRVCQS5N5N34Y6PGUJ5FPWOCHHO5KMU2PJ5CUXWTHW7RQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504844445 (203h25m3.935924566s) [noise=true , address=45.91.116.12:4228 , piece_id=X5VGNN2WVGEW3E4MPUO5MQWZ7L2JMB345IQUUTBI4DYW5HESK33A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504957571 (203h25m3.936037702s) [noise=true , address=104.36.82.23:4475 , piece_id=TYHG2VUDQ4RRMFLUOLV3U73G7JXA6ZRC757TD34Q2MZTJNRMUIZA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505108992 (203h25m3.936189123s) [noise=true , address=104.36.82.59:4331 , piece_id=SPNJSZRY7LNIS7ZCHJLOOWKVZU7AP3CODFM73CGOLQZU7HUCT2IQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505263607 (203h25m3.936343738s) [noise=true , address=66.23.193.175:4553 , piece_id=BXVNQMDXCQPEUSLIZ24VI22F7FOOJN2MEGJU6UI2SDQIZJSFQJ2A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505341626 (203h25m3.936421757s) [noise=true , address=77.90.31.10:4523 , piece_id=22C56K4AOQCL5N6WR7QECVAAXHOS3OUNMGWCRFHP5GKSTHIT7OLA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505565765 (203h25m3.936645896s) [noise=true , address=104.36.82.4:4836 , piece_id=JJWCMFYVQUU6WGEVBEJS4JPZQNNZERI5BVTBBAQQUZV3WD63NGIA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505597326 (203h25m3.936677457s) [noise=true , address=77.90.31.11:4563 , piece_id=QDC72JPWTU4LH5URAU44ECPV72L4NQPWGDDOHHVYSGNJTFS7C6XQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505853546 (203h25m3.936933677s) [noise=true , address=104.36.82.18:4396 , piece_id=3CORBTMNXLB5AG6ZGAPLPBGGFEQ4SSMJHRGXPESWTJD5YPYVUKFA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505894053 (203h25m3.936974184s) [noise=true , address=104.36.82.44:4896 , piece_id=2DQKKZFJX4N6W5UOOYX4QWSZ3GAFBMQIIDCKU3N3N5MLM4Q7T2AQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506090339 (203h25m3.937170471s) [noise=true , address=77.90.31.13:4658 , piece_id=XONXAMV3VAHHSHNIDMAIMMD6KOCKLYJQAUTNVMSUXB6ZCWP5THDA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506165653 (203h25m3.937245784s) [noise=true , address=104.36.82.5:4850 , piece_id=6ZYI2JS5HMQKTUM2XR4ZTSJMC3MHVILV4YOQ7ANA5PX3FDJXJ2DA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506275653 (203h25m3.937355774s) [noise=true , address=45.91.116.14:4199 , piece_id=HOTL5QUTJLBBZNF6A4WPRKMQXB72I3TUVSDYUVWMU7G4ZIXOIK3A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506396365 (203h25m3.937476496s) [noise=true , address=104.36.82.34:4565 , piece_id=LUXKT7N4OUT4UJA345KTE2T3QUD3LEGLUQKQTDZADYBOGE2Z5XNQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506467330 (203h25m3.937547461s) [noise=true , address=104.36.82.60:4354 , piece_id=M3INHRZ2XHGLNFSI7OHKHKNCVTSOWTSCJJCGHTTYCAJGFLDWLL7A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:10.617749206 (203h25m5.048829337s) [upload=211594]: Next returned nil segment Aug 17 04:51:10.617765637 (203h25m5.048845758s) [upload=211594]: Waiting for error group managing segments... Aug 17 04:51:10.899081476 (203h25m5.330161607s) [noise=true , address=64.49.13.169:4706 , piece_id=Y4SS3PCBRVCQS5N5N34Y6PGUJ5FPWOCHHO5KMU2PJ5CUXWTHW7RQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:10.959773754 (203h25m5.390853875s) [noise=true , address=66.23.193.175:4553 , piece_id=BXVNQMDXCQPEUSLIZ24VI22F7FOOJN2MEGJU6UI2SDQIZJSFQJ2A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.387426163 (203h25m5.818506294s) [noise=true , address=77.90.31.11:4563 , piece_id=QDC72JPWTU4LH5URAU44ECPV72L4NQPWGDDOHHVYSGNJTFS7C6XQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.392712709 (203h25m5.82379284s) [noise=true , address=45.91.116.14:4199 , piece_id=HOTL5QUTJLBBZNF6A4WPRKMQXB72I3TUVSDYUVWMU7G4ZIXOIK3A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.399286516 (203h25m5.830366637s) [noise=true , address=104.36.82.4:4836 , piece_id=JJWCMFYVQUU6WGEVBEJS4JPZQNNZERI5BVTBBAQQUZV3WD63NGIA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.412750859 (203h25m5.84383099s) [noise=true , address=104.36.82.23:4475 , piece_id=TYHG2VUDQ4RRMFLUOLV3U73G7JXA6ZRC757TD34Q2MZTJNRMUIZA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.439625902 (203h25m5.870706033s) [noise=true , address=77.90.31.10:4523 , piece_id=22C56K4AOQCL5N6WR7QECVAAXHOS3OUNMGWCRFHP5GKSTHIT7OLA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.465577498 (203h25m5.896657629s) [noise=true , address=104.36.82.34:4565 , piece_id=LUXKT7N4OUT4UJA345KTE2T3QUD3LEGLUQKQTDZADYBOGE2Z5XNQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.469847087 (203h25m5.900927218s) [noise=true , address=104.36.82.60:4354 , piece_id=M3INHRZ2XHGLNFSI7OHKHKNCVTSOWTSCJJCGHTTYCAJGFLDWLL7A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.470697434 (203h25m5.901777555s) [noise=true , address=104.36.82.5:4850 , piece_id=6ZYI2JS5HMQKTUM2XR4ZTSJMC3MHVILV4YOQ7ANA5PX3FDJXJ2DA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.470949846 (203h25m5.902029978s) [noise=true , address=104.36.82.19:4436 , piece_id=VN6NIXXOT6KUQE3JU6X35TPYNCI5K4KOSBLVXMOKSP6N2SYRFM6Q, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.486560086 (203h25m5.917640217s) [noise=true , address=104.36.82.2:4748 , piece_id=T6MOSFTUKYGVT2YSCEGXMSDXLULB6NI4RGGGB3SSARO7ALNZJN5Q, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.498376395 (203h25m5.929456526s) [noise=true , address=104.36.82.44:4896 , piece_id=2DQKKZFJX4N6W5UOOYX4QWSZ3GAFBMQIIDCKU3N3N5MLM4Q7T2AQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.512939881 (203h25m5.944020012s) [noise=true , address=45.91.116.12:4228 , piece_id=X5VGNN2WVGEW3E4MPUO5MQWZ7L2JMB345IQUUTBI4DYW5HESK33A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.512969688 (203h25m5.944049819s) [seg_pos={0 0}, upload=211594]: Segment reached optimal threshold of 14 pieces. Aug 17 04:51:11.513156405 (203h25m5.944236536s) [noise=true , address=77.90.31.13:4658 , piece_id=XONXAMV3VAHHSHNIDMAIMMD6KOCKLYJQAUTNVMSUXB6ZCWP5THDA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129aongkQE9HMNz8vsNNA5SvKbhsHqdt9UrtSTPJJpP9mkFsboX): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513220778 (203h25m5.944300909s) [noise=true , address=104.36.82.58:4295 , piece_id=3LNVXIW5IU2BQZRH6QAVV7A2PJ3L3YSGNOXBMIYOJMFBVHJZUPYA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oLa34qwPBAx3CTvYYj7GN3oxxWGJ9dAYXfke1eUftZc6xzfqg): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513269471 (203h25m5.944349603s) [noise=true , address=104.36.82.59:4331 , piece_id=SPNJSZRY7LNIS7ZCHJLOOWKVZU7AP3CODFM73CGOLQZU7HUCT2IQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513316901 (203h25m5.944397033s) [noise=true , address=104.36.82.18:4396 , piece_id=3CORBTMNXLB5AG6ZGAPLPBGGFEQ4SSMJHRGXPESWTJD5YPYVUKFA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1k5gzJBGWhpARcfiynburgfTqJ2t7sVNs1mAkUfToMG5saxPqx): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513333754 (203h25m5.944413885s) [seg_pos={0 0}, upload=211594]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 04:51:11.513406654 (203h25m5.944486775s) [upload=211594]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 04:51:11.513424287 (203h25m5.944504418s) [upload=211594]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:51:11.809871930 (203h25m6.240952061s) [upload=211594]: Done uploading segments... Aug 17 04:51:11.809892119 (203h25m6.24097225s) [upload=211594]: Upload finished. err: <nil> Duration: 5.567072379s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................XXX........................................] duration: 5.535167864s FAIL (2025-08-17 04:52:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== g_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761278730 (202h56m21.192358861s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761661883 (202h56m21.192742014s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761716548 (202h56m21.192796679s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761972016 (202h56m21.193052147s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762254878 (202h56m21.193335009s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762523682 (202h56m21.193603803s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762864294 (202h56m21.193944415s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763179537 (202h56m21.194259668s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763478499 (202h56m21.19455864s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763757683 (202h56m21.194837814s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763988195 (202h56m21.195068326s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764273320 (202h56m21.195353451s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764555100 (202h56m21.195635221s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764757416 (202h56m21.195837547s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.765868993 (202h56m21.196949124s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.767152809 (202h56m21.19823294s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.768577886 (202h56m21.199658017s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:27.934309011 (202h56m22.365389142s) [upload=211101]: Next returned nil segment Aug 17 04:22:27.934342545 (202h56m22.365422666s) [upload=211101]: Waiting for error group managing segments... Aug 17 04:22:28.231985083 (202h56m22.663065214s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.670701256 (202h56m23.101781427s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.671952431 (202h56m23.103032562s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.698468278 (202h56m23.129548409s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.720171187 (202h56m23.151251318s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.731591429 (202h56m23.16267156s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.798148986 (202h56m23.229229117s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.804849037 (202h56m23.235929168s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.833066822 (202h56m23.264147013s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.846684710 (202h56m23.277764831s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:23:25.582810947 (202h57m20.013891078s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12TU4zvu38mF1fNSergCtMj2cK9eu57Rdfpt5MVk3tc4MZF6dNr, address:104.36.82.34:4558): protocol: rpc: dial tcp 104.36.82.34:4558: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582856475 (202h57m20.013936606s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12D9nh4XFmsjjFcZEMjaL4uw4Whsa8nfMrAsYPu6NVpsxgb3jvf, address:104.36.82.35:4594): protocol: rpc: dial tcp 104.36.82.35:4594: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582973238 (202h57m20.014053369s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1WvyEw8XevCiQ8yRgzyMo7WTVzpEJ158ErKfAeV4dSTBkS1oRF, address:104.36.82.19:4436): protocol: rpc: dial tcp 104.36.82.19:4436: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583033393 (202h57m20.014113524s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:104.36.82.58:4283): protocol: rpc: dial tcp 104.36.82.58:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583069241 (202h57m20.014149362s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:126Z6tvp49dLLdj3T5snJc4tBVGzcX5UBaGafFN3PkhF92vxpfs, address:104.36.82.50:4047): protocol: rpc: dial tcp 104.36.82.50:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583076716 (202h57m20.014156837s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1zzW2z6noGQT6xdSmwSQBNYwip6gZjnmqZg7qdb6BP3nHyCLFz, address:104.36.82.39:4753): protocol: rpc: dial tcp 104.36.82.39:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583107535 (202h57m20.014187666s) [noise=true , address=104.36.82.23:4474 , piece_id=RITPHITVW655UQETT6UKS7BZO2S3F4P2MJIO4REOUKYYP4E2CDVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 188.166.225.170:36642->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583165355 (202h57m20.014245487s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh, address:104.36.82.5:4855): protocol: write tcp 188.166.225.170:47052->104.36.82.5:4855: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583194000 (202h57m20.014274131s) [seg_pos={0 0}, upload=211101]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:25.810738626 (202h57m20.241818757s) [upload=211101]: Done uploading segments... Aug 17 04:23:25.810761991 (202h57m20.241842122s) [upload=211101]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.535213812s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X.....X................] duration: 2.107601449s FAIL (2025-08-17 04:52:01 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 17 04:41:26.040732864 (203h15m20.471812985s) [upload=211420]: Starting upload Aug 17 04:41:26.040780384 (203h15m20.471860515s) [upload=211420]: Done starting upload Aug 17 04:41:26.040828978 (203h15m20.471909109s) [upload=211420]: Uploading segments... Aug 17 04:41:26.040863473 (203h15m20.471943604s) [upload=211420]: Got next segment. Inline: true Aug 17 04:41:26.040867290 (203h15m20.471947422s) [upload=211420]: Waiting for error group managing segments... Aug 17 04:41:26.040897880 (203h15m20.471978011s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:41:26.040901597 (203h15m20.471981728s) [upload=211420]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:41:26.040904041 (203h15m20.471984162s) [upload=211420]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:41:26.462140724 (203h15m20.893220855s) [upload=211420]: Done uploading segments... Aug 17 04:41:26.462159310 (203h15m20.893239431s) [upload=211420]: Upload finished. err: <nil> Duration:* 2.107637798s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [......................XX....................................] duration: 6.689135421s FAIL (2025-08-17 04:51:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.689156462s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [................XXXX........................................] duration: 6.176503115s FAIL (2025-08-17 04:51:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.176531109s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 10 out of 60 is failed (threshold=3) [..........X..........XXXX.X.X.......X...X.................X.] duration: 5.660855373s FAIL (2025-08-17 04:51:43 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 04:51:09.056938814 (203h25m3.488018935s) [upload=211594]: Starting upload Aug 17 04:51:09.057017635 (203h25m3.488097766s) [upload=211594]: Done starting upload Aug 17 04:51:09.057190195 (203h25m3.488270326s) [upload=211594]: Uploading segments... Aug 17 04:51:09.057308371 (203h25m3.488388502s) [upload=211594]: Got next segment. Inline: false Aug 17 04:51:09.057342866 (203h25m3.488422997s) [upload=211594]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:51:09.057347857 (203h25m3.488427978s) [upload=211594]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:51:09.504198039 (203h25m3.93527816s) [seg_pos={0 0}, upload=211594]: Begin upload segment... Aug 17 04:51:09.504246441 (203h25m3.935326572s) [seg_pos={0 0}, upload=211594]: Done begin upload segment. Aug 17 04:51:09.504264686 (203h25m3.935344817s) [noise=true , address=104.36.82.2:4748 , piece_id=T6MOSFTUKYGVT2YSCEGXMSDXLULB6NI4RGGGB3SSARO7ALNZJN5Q, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504308099 (203h25m3.93538823s) [noise=true , address=104.36.82.19:4436 , piece_id=VN6NIXXOT6KUQE3JU6X35TPYNCI5K4KOSBLVXMOKSP6N2SYRFM6Q, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504588896 (203h25m3.935669017s) [noise=true , address=104.36.82.58:4295 , piece_id=3LNVXIW5IU2BQZRH6QAVV7A2PJ3L3YSGNOXBMIYOJMFBVHJZUPYA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504650284 (203h25m3.935730405s) [noise=true , address=64.49.13.169:4706 , piece_id=Y4SS3PCBRVCQS5N5N34Y6PGUJ5FPWOCHHO5KMU2PJ5CUXWTHW7RQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504844445 (203h25m3.935924566s) [noise=true , address=45.91.116.12:4228 , piece_id=X5VGNN2WVGEW3E4MPUO5MQWZ7L2JMB345IQUUTBI4DYW5HESK33A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.504957571 (203h25m3.936037702s) [noise=true , address=104.36.82.23:4475 , piece_id=TYHG2VUDQ4RRMFLUOLV3U73G7JXA6ZRC757TD34Q2MZTJNRMUIZA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505108992 (203h25m3.936189123s) [noise=true , address=104.36.82.59:4331 , piece_id=SPNJSZRY7LNIS7ZCHJLOOWKVZU7AP3CODFM73CGOLQZU7HUCT2IQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505263607 (203h25m3.936343738s) [noise=true , address=66.23.193.175:4553 , piece_id=BXVNQMDXCQPEUSLIZ24VI22F7FOOJN2MEGJU6UI2SDQIZJSFQJ2A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505341626 (203h25m3.936421757s) [noise=true , address=77.90.31.10:4523 , piece_id=22C56K4AOQCL5N6WR7QECVAAXHOS3OUNMGWCRFHP5GKSTHIT7OLA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505565765 (203h25m3.936645896s) [noise=true , address=104.36.82.4:4836 , piece_id=JJWCMFYVQUU6WGEVBEJS4JPZQNNZERI5BVTBBAQQUZV3WD63NGIA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505597326 (203h25m3.936677457s) [noise=true , address=77.90.31.11:4563 , piece_id=QDC72JPWTU4LH5URAU44ECPV72L4NQPWGDDOHHVYSGNJTFS7C6XQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505853546 (203h25m3.936933677s) [noise=true , address=104.36.82.18:4396 , piece_id=3CORBTMNXLB5AG6ZGAPLPBGGFEQ4SSMJHRGXPESWTJD5YPYVUKFA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.505894053 (203h25m3.936974184s) [noise=true , address=104.36.82.44:4896 , piece_id=2DQKKZFJX4N6W5UOOYX4QWSZ3GAFBMQIIDCKU3N3N5MLM4Q7T2AQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506090339 (203h25m3.937170471s) [noise=true , address=77.90.31.13:4658 , piece_id=XONXAMV3VAHHSHNIDMAIMMD6KOCKLYJQAUTNVMSUXB6ZCWP5THDA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506165653 (203h25m3.937245784s) [noise=true , address=104.36.82.5:4850 , piece_id=6ZYI2JS5HMQKTUM2XR4ZTSJMC3MHVILV4YOQ7ANA5PX3FDJXJ2DA, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506275653 (203h25m3.937355774s) [noise=true , address=45.91.116.14:4199 , piece_id=HOTL5QUTJLBBZNF6A4WPRKMQXB72I3TUVSDYUVWMU7G4ZIXOIK3A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506396365 (203h25m3.937476496s) [noise=true , address=104.36.82.34:4565 , piece_id=LUXKT7N4OUT4UJA345KTE2T3QUD3LEGLUQKQTDZADYBOGE2Z5XNQ, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:09.506467330 (203h25m3.937547461s) [noise=true , address=104.36.82.60:4354 , piece_id=M3INHRZ2XHGLNFSI7OHKHKNCVTSOWTSCJJCGHTTYCAJGFLDWLL7A, seg_pos={0 0}, upload=211594]: Uploading piece... Aug 17 04:51:10.617749206 (203h25m5.048829337s) [upload=211594]: Next returned nil segment Aug 17 04:51:10.617765637 (203h25m5.048845758s) [upload=211594]: Waiting for error group managing segments... Aug 17 04:51:10.899081476 (203h25m5.330161607s) [noise=true , address=64.49.13.169:4706 , piece_id=Y4SS3PCBRVCQS5N5N34Y6PGUJ5FPWOCHHO5KMU2PJ5CUXWTHW7RQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:10.959773754 (203h25m5.390853875s) [noise=true , address=66.23.193.175:4553 , piece_id=BXVNQMDXCQPEUSLIZ24VI22F7FOOJN2MEGJU6UI2SDQIZJSFQJ2A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.387426163 (203h25m5.818506294s) [noise=true , address=77.90.31.11:4563 , piece_id=QDC72JPWTU4LH5URAU44ECPV72L4NQPWGDDOHHVYSGNJTFS7C6XQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.392712709 (203h25m5.82379284s) [noise=true , address=45.91.116.14:4199 , piece_id=HOTL5QUTJLBBZNF6A4WPRKMQXB72I3TUVSDYUVWMU7G4ZIXOIK3A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.399286516 (203h25m5.830366637s) [noise=true , address=104.36.82.4:4836 , piece_id=JJWCMFYVQUU6WGEVBEJS4JPZQNNZERI5BVTBBAQQUZV3WD63NGIA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.412750859 (203h25m5.84383099s) [noise=true , address=104.36.82.23:4475 , piece_id=TYHG2VUDQ4RRMFLUOLV3U73G7JXA6ZRC757TD34Q2MZTJNRMUIZA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.439625902 (203h25m5.870706033s) [noise=true , address=77.90.31.10:4523 , piece_id=22C56K4AOQCL5N6WR7QECVAAXHOS3OUNMGWCRFHP5GKSTHIT7OLA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.465577498 (203h25m5.896657629s) [noise=true , address=104.36.82.34:4565 , piece_id=LUXKT7N4OUT4UJA345KTE2T3QUD3LEGLUQKQTDZADYBOGE2Z5XNQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.469847087 (203h25m5.900927218s) [noise=true , address=104.36.82.60:4354 , piece_id=M3INHRZ2XHGLNFSI7OHKHKNCVTSOWTSCJJCGHTTYCAJGFLDWLL7A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.470697434 (203h25m5.901777555s) [noise=true , address=104.36.82.5:4850 , piece_id=6ZYI2JS5HMQKTUM2XR4ZTSJMC3MHVILV4YOQ7ANA5PX3FDJXJ2DA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.470949846 (203h25m5.902029978s) [noise=true , address=104.36.82.19:4436 , piece_id=VN6NIXXOT6KUQE3JU6X35TPYNCI5K4KOSBLVXMOKSP6N2SYRFM6Q, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.486560086 (203h25m5.917640217s) [noise=true , address=104.36.82.2:4748 , piece_id=T6MOSFTUKYGVT2YSCEGXMSDXLULB6NI4RGGGB3SSARO7ALNZJN5Q, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.498376395 (203h25m5.929456526s) [noise=true , address=104.36.82.44:4896 , piece_id=2DQKKZFJX4N6W5UOOYX4QWSZ3GAFBMQIIDCKU3N3N5MLM4Q7T2AQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.512939881 (203h25m5.944020012s) [noise=true , address=45.91.116.12:4228 , piece_id=X5VGNN2WVGEW3E4MPUO5MQWZ7L2JMB345IQUUTBI4DYW5HESK33A, seg_pos={0 0}, upload=211594]: Done uploading piece. err: <nil> Aug 17 04:51:11.512969688 (203h25m5.944049819s) [seg_pos={0 0}, upload=211594]: Segment reached optimal threshold of 14 pieces. Aug 17 04:51:11.513156405 (203h25m5.944236536s) [noise=true , address=77.90.31.13:4658 , piece_id=XONXAMV3VAHHSHNIDMAIMMD6KOCKLYJQAUTNVMSUXB6ZCWP5THDA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129aongkQE9HMNz8vsNNA5SvKbhsHqdt9UrtSTPJJpP9mkFsboX): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513220778 (203h25m5.944300909s) [noise=true , address=104.36.82.58:4295 , piece_id=3LNVXIW5IU2BQZRH6QAVV7A2PJ3L3YSGNOXBMIYOJMFBVHJZUPYA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oLa34qwPBAx3CTvYYj7GN3oxxWGJ9dAYXfke1eUftZc6xzfqg): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513269471 (203h25m5.944349603s) [noise=true , address=104.36.82.59:4331 , piece_id=SPNJSZRY7LNIS7ZCHJLOOWKVZU7AP3CODFM73CGOLQZU7HUCT2IQ, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513316901 (203h25m5.944397033s) [noise=true , address=104.36.82.18:4396 , piece_id=3CORBTMNXLB5AG6ZGAPLPBGGFEQ4SSMJHRGXPESWTJD5YPYVUKFA, seg_pos={0 0}, upload=211594]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1k5gzJBGWhpARcfiynburgfTqJ2t7sVNs1mAkUfToMG5saxPqx): protocol: expected piece hash; context canceled; EOF Aug 17 04:51:11.513333754 (203h25m5.944413885s) [seg_pos={0 0}, upload=211594]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 04:51:11.513406654 (203h25m5.944486775s) [upload=211594]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 04:51:11.513424287 (203h25m5.944504418s) [upload=211594]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:51:11.809871930 (203h25m6.240952061s) [upload=211594]: Done uploading segments... Aug 17 04:51:11.809892119 (203h25m6.24097225s) [upload=211594]: Upload finished. err: <nil> Duration: 5.660876483s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)