Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............X...........................................X.] duration: 1.829182955s FAIL (2025-08-10 23:04:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 87485733ea572aac21ef178fe70f9ccd01606e1ec8b031e2feb64d610db79d6c != before d29ad4447e684ddad18d87c11fbeaf3ef3aa3d31c4bd22ab9273dee072942636 ======LOGS====== Aug 10 23:03:42.181544880 (53h37m36.612625011s) [upload=55918]: Starting upload Aug 10 23:03:42.181592230 (53h37m36.612672351s) [upload=55918]: Done starting upload Aug 10 23:03:42.181644050 (53h37m36.612724171s) [upload=55918]: Uploading segments... Aug 10 23:03:42.181683094 (53h37m36.612763225s) [upload=55918]: Got next segment. Inline: true Aug 10 23:03:42.181687823 (53h37m36.612767954s) [upload=55918]: Waiting for error group managing segments... Aug 10 23:03:42.181727299 (53h37m36.61280742s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:03:42.181731487 (53h37m36.612811618s) [upload=55918]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 23:03:42.181734372 (53h37m36.612814503s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:03:42.463145140 (53h37m36.894225271s) [upload=55918]: Done uploading segments... Aug 10 23:03:42.463162954 (53h37m36.894243075s) [upload=55918]: Upload finished. err: <nil> Duration:* 1.8292117s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [.............X.......X.X.....X.X...........XX...............] duration: 10.314889536s FAIL (2025-08-10 23:04:14 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 22:52:30.226093181 (53h26m24.657173312s) [upload=55724]: Starting upload Aug 10 22:52:30.226146663 (53h26m24.657226794s) [upload=55724]: Done starting upload Aug 10 22:52:30.226228670 (53h26m24.657308801s) [upload=55724]: Uploading segments... Aug 10 22:52:30.226383958 (53h26m24.657464089s) [upload=55724]: Got next segment. Inline: false Aug 10 22:52:30.226415327 (53h26m24.657495458s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:52:30.226421208 (53h26m24.65750134s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 22:52:30.677132253 (53h26m25.108212384s) [seg_pos={0 0}, upload=55724]: Begin upload segment... Aug 10 22:52:30.677190715 (53h26m25.108270846s) [seg_pos={0 0}, upload=55724]: Done begin upload segment. Aug 10 22:52:30.677225091 (53h26m25.108305212s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677252474 (53h26m25.108332605s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677613114 (53h26m25.108693245s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678313253 (53h26m25.109393384s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678331708 (53h26m25.109411839s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678613808 (53h26m25.109693939s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678652442 (53h26m25.109732573s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678866331 (53h26m25.109946462s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679169000 (53h26m25.110249131s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679440199 (53h26m25.11052033s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679713732 (53h26m25.110793863s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679926259 (53h26m25.11100639s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680126402 (53h26m25.111206533s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680326895 (53h26m25.111407026s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680530465 (53h26m25.111610596s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680733313 (53h26m25.111813444s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681408626 (53h26m25.112488777s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681726623 (53h26m25.112806755s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:31.876216927 (53h26m26.307297058s) [upload=55724]: Next returned nil segment Aug 10 22:52:31.876232165 (53h26m26.307312296s) [upload=55724]: Waiting for error group managing segments... Aug 10 22:52:32.182969622 (53h26m26.614049743s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.631903297 (53h26m27.062983428s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.652429340 (53h26m27.083509471s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.695199823 (53h26m27.126279954s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.740711713 (53h26m27.171791844s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.776799260 (53h26m27.207879391s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.823959072 (53h26m27.255039203s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.873921155 (53h26m27.305001286s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886211631 (53h26m27.317291762s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886240236 (53h26m27.317320367s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:33.361231516 (53h26m27.792311647s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.095423782 (53h26m28.526503913s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.191766598 (53h26m28.622846729s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957234535 (53h26m29.388314666s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957270263 (53h26m29.388350384s) [seg_pos={0 0}, upload=55724]: Segment reached optimal threshold of 14 pieces. Aug 10 22:52:34.957519320 (53h26m29.388599441s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ezboNAWPbaK9UDENYoBb3iHjNU2ahn7mikiteN7y3Aj6uw4Nx): protocol: context canceled; context canceled Aug 10 22:52:34.957638617 (53h26m29.388718748s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SwpAaXTA3BqUvwubzmJkvwokdz6SSpT9t8HPtQcXXJxmcgfm2): protocol: read tcp 188.166.225.170:38236->104.36.82.37:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957696528 (53h26m29.388776659s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2euZtUdrPZGo2PieqDnKtSftgtfdTRk6V8VfYS4yUW5z3qM5): protocol: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection Aug 10 22:52:34.957729272 (53h26m29.388809393s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tvoTD1LoSj3q2rLiDckfT56dXNKKwdVu1ZAsBZExNFVTom9tH): protocol: rpc: dial tcp 104.36.82.8:4995: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957748227 (53h26m29.388828348s) [seg_pos={0 0}, upload=55724]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 22:52:34.957846646 (53h26m29.388926767s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 22:52:34.957851144 (53h26m29.388931275s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:52:35.306840333 (53h26m29.737920484s) [upload=55724]: Done uploading segments... Aug 10 22:52:35.306857145 (53h26m29.737937276s) [upload=55724]: Upload finished. err: <nil> Duration: 10.314916596s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................XX..................................] duration: 7.809565341s FAIL (2025-08-10 23:03:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 7.809594026s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............X...........................................X] duration: 1.667910718s FAIL (2025-08-10 23:03:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 87485733ea572aac21ef178fe70f9ccd01606e1ec8b031e2feb64d610db79d6c != before d29ad4447e684ddad18d87c11fbeaf3ef3aa3d31c4bd22ab9273dee072942636 ======LOGS====== Aug 10 23:03:42.181544880 (53h37m36.612625011s) [upload=55918]: Starting upload Aug 10 23:03:42.181592230 (53h37m36.612672351s) [upload=55918]: Done starting upload Aug 10 23:03:42.181644050 (53h37m36.612724171s) [upload=55918]: Uploading segments... Aug 10 23:03:42.181683094 (53h37m36.612763225s) [upload=55918]: Got next segment. Inline: true Aug 10 23:03:42.181687823 (53h37m36.612767954s) [upload=55918]: Waiting for error group managing segments... Aug 10 23:03:42.181727299 (53h37m36.61280742s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 23:03:42.181731487 (53h37m36.612811618s) [upload=55918]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 23:03:42.181734372 (53h37m36.612814503s) [upload=55918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 23:03:42.463145140 (53h37m36.894225271s) [upload=55918]: Done uploading segments... Aug 10 23:03:42.463162954 (53h37m36.894243075s) [upload=55918]: Upload finished. err: <nil> Duration:* 1.667957507s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [..............X.......X.X.....X.X...........XX..............] duration: 7.288233353s FAIL (2025-08-10 23:03:37 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 22:52:30.226093181 (53h26m24.657173312s) [upload=55724]: Starting upload Aug 10 22:52:30.226146663 (53h26m24.657226794s) [upload=55724]: Done starting upload Aug 10 22:52:30.226228670 (53h26m24.657308801s) [upload=55724]: Uploading segments... Aug 10 22:52:30.226383958 (53h26m24.657464089s) [upload=55724]: Got next segment. Inline: false Aug 10 22:52:30.226415327 (53h26m24.657495458s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:52:30.226421208 (53h26m24.65750134s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 22:52:30.677132253 (53h26m25.108212384s) [seg_pos={0 0}, upload=55724]: Begin upload segment... Aug 10 22:52:30.677190715 (53h26m25.108270846s) [seg_pos={0 0}, upload=55724]: Done begin upload segment. Aug 10 22:52:30.677225091 (53h26m25.108305212s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677252474 (53h26m25.108332605s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677613114 (53h26m25.108693245s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678313253 (53h26m25.109393384s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678331708 (53h26m25.109411839s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678613808 (53h26m25.109693939s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678652442 (53h26m25.109732573s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678866331 (53h26m25.109946462s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679169000 (53h26m25.110249131s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679440199 (53h26m25.11052033s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679713732 (53h26m25.110793863s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679926259 (53h26m25.11100639s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680126402 (53h26m25.111206533s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680326895 (53h26m25.111407026s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680530465 (53h26m25.111610596s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680733313 (53h26m25.111813444s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681408626 (53h26m25.112488777s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681726623 (53h26m25.112806755s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:31.876216927 (53h26m26.307297058s) [upload=55724]: Next returned nil segment Aug 10 22:52:31.876232165 (53h26m26.307312296s) [upload=55724]: Waiting for error group managing segments... Aug 10 22:52:32.182969622 (53h26m26.614049743s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.631903297 (53h26m27.062983428s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.652429340 (53h26m27.083509471s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.695199823 (53h26m27.126279954s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.740711713 (53h26m27.171791844s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.776799260 (53h26m27.207879391s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.823959072 (53h26m27.255039203s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.873921155 (53h26m27.305001286s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886211631 (53h26m27.317291762s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886240236 (53h26m27.317320367s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:33.361231516 (53h26m27.792311647s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.095423782 (53h26m28.526503913s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.191766598 (53h26m28.622846729s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957234535 (53h26m29.388314666s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957270263 (53h26m29.388350384s) [seg_pos={0 0}, upload=55724]: Segment reached optimal threshold of 14 pieces. Aug 10 22:52:34.957519320 (53h26m29.388599441s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ezboNAWPbaK9UDENYoBb3iHjNU2ahn7mikiteN7y3Aj6uw4Nx): protocol: context canceled; context canceled Aug 10 22:52:34.957638617 (53h26m29.388718748s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SwpAaXTA3BqUvwubzmJkvwokdz6SSpT9t8HPtQcXXJxmcgfm2): protocol: read tcp 188.166.225.170:38236->104.36.82.37:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957696528 (53h26m29.388776659s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2euZtUdrPZGo2PieqDnKtSftgtfdTRk6V8VfYS4yUW5z3qM5): protocol: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection Aug 10 22:52:34.957729272 (53h26m29.388809393s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tvoTD1LoSj3q2rLiDckfT56dXNKKwdVu1ZAsBZExNFVTom9tH): protocol: rpc: dial tcp 104.36.82.8:4995: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957748227 (53h26m29.388828348s) [seg_pos={0 0}, upload=55724]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 22:52:34.957846646 (53h26m29.388926767s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 22:52:34.957851144 (53h26m29.388931275s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:52:35.306840333 (53h26m29.737920484s) [upload=55724]: Done uploading segments... Aug 10 22:52:35.306857145 (53h26m29.737937276s) [upload=55724]: Upload finished. err: <nil> Duration: 7.288256837s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................XX.................................] duration: 7.511406945s FAIL (2025-08-10 23:03:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 7.511431803s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.467528582s FAIL (2025-08-10 23:03:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 6.46755861s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 1.828855228s FAIL (2025-08-10 23:03:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 10 22:33:23.090759975 (53h7m17.521840126s) [upload=55388]: Starting upload Aug 10 22:33:23.090831261 (53h7m17.521911422s) [upload=55388]: Done starting upload Aug 10 22:33:23.090903269 (53h7m17.52198342s) [upload=55388]: Uploading segments... Aug 10 22:33:23.090955248 (53h7m17.52203541s) [upload=55388]: Got next segment. Inline: true Aug 10 22:33:23.090961811 (53h7m17.522041962s) [upload=55388]: Waiting for error group managing segments... Aug 10 22:33:23.091016636 (53h7m17.522096807s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:33:23.091023408 (53h7m17.52210356s) [upload=55388]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 22:33:23.091028138 (53h7m17.522108289s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:33:23.358287015 (53h7m17.789367146s) [upload=55388]: Done uploading segments... Aug 10 22:33:23.358303086 (53h7m17.789383207s) [upload=55388]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.828883902s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.274954574s FAIL (2025-08-10 23:02:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 6.274980964s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 1.906502604s FAIL (2025-08-10 23:02:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 10 22:33:23.090759975 (53h7m17.521840126s) [upload=55388]: Starting upload Aug 10 22:33:23.090831261 (53h7m17.521911422s) [upload=55388]: Done starting upload Aug 10 22:33:23.090903269 (53h7m17.52198342s) [upload=55388]: Uploading segments... Aug 10 22:33:23.090955248 (53h7m17.52203541s) [upload=55388]: Got next segment. Inline: true Aug 10 22:33:23.090961811 (53h7m17.522041962s) [upload=55388]: Waiting for error group managing segments... Aug 10 22:33:23.091016636 (53h7m17.522096807s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:33:23.091023408 (53h7m17.52210356s) [upload=55388]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 22:33:23.091028138 (53h7m17.522108289s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:33:23.358287015 (53h7m17.789367146s) [upload=55388]: Done uploading segments... Aug 10 22:33:23.358303086 (53h7m17.789383207s) [upload=55388]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.906532992s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................XX................................] duration: 7.318016508s FAIL (2025-08-10 23:02:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 7.318043931s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 7.339176181s FAIL (2025-08-10 23:02:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 7.339205718s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [...............X.......X.X.....X.X...........XX.............] duration: 8.309667015s FAIL (2025-08-10 23:02:35 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 22:52:30.226093181 (53h26m24.657173312s) [upload=55724]: Starting upload Aug 10 22:52:30.226146663 (53h26m24.657226794s) [upload=55724]: Done starting upload Aug 10 22:52:30.226228670 (53h26m24.657308801s) [upload=55724]: Uploading segments... Aug 10 22:52:30.226383958 (53h26m24.657464089s) [upload=55724]: Got next segment. Inline: false Aug 10 22:52:30.226415327 (53h26m24.657495458s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:52:30.226421208 (53h26m24.65750134s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 22:52:30.677132253 (53h26m25.108212384s) [seg_pos={0 0}, upload=55724]: Begin upload segment... Aug 10 22:52:30.677190715 (53h26m25.108270846s) [seg_pos={0 0}, upload=55724]: Done begin upload segment. Aug 10 22:52:30.677225091 (53h26m25.108305212s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677252474 (53h26m25.108332605s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677613114 (53h26m25.108693245s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678313253 (53h26m25.109393384s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678331708 (53h26m25.109411839s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678613808 (53h26m25.109693939s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678652442 (53h26m25.109732573s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678866331 (53h26m25.109946462s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679169000 (53h26m25.110249131s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679440199 (53h26m25.11052033s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679713732 (53h26m25.110793863s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679926259 (53h26m25.11100639s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680126402 (53h26m25.111206533s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680326895 (53h26m25.111407026s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680530465 (53h26m25.111610596s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680733313 (53h26m25.111813444s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681408626 (53h26m25.112488777s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681726623 (53h26m25.112806755s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:31.876216927 (53h26m26.307297058s) [upload=55724]: Next returned nil segment Aug 10 22:52:31.876232165 (53h26m26.307312296s) [upload=55724]: Waiting for error group managing segments... Aug 10 22:52:32.182969622 (53h26m26.614049743s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.631903297 (53h26m27.062983428s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.652429340 (53h26m27.083509471s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.695199823 (53h26m27.126279954s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.740711713 (53h26m27.171791844s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.776799260 (53h26m27.207879391s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.823959072 (53h26m27.255039203s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.873921155 (53h26m27.305001286s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886211631 (53h26m27.317291762s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886240236 (53h26m27.317320367s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:33.361231516 (53h26m27.792311647s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.095423782 (53h26m28.526503913s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.191766598 (53h26m28.622846729s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957234535 (53h26m29.388314666s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957270263 (53h26m29.388350384s) [seg_pos={0 0}, upload=55724]: Segment reached optimal threshold of 14 pieces. Aug 10 22:52:34.957519320 (53h26m29.388599441s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ezboNAWPbaK9UDENYoBb3iHjNU2ahn7mikiteN7y3Aj6uw4Nx): protocol: context canceled; context canceled Aug 10 22:52:34.957638617 (53h26m29.388718748s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SwpAaXTA3BqUvwubzmJkvwokdz6SSpT9t8HPtQcXXJxmcgfm2): protocol: read tcp 188.166.225.170:38236->104.36.82.37:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957696528 (53h26m29.388776659s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2euZtUdrPZGo2PieqDnKtSftgtfdTRk6V8VfYS4yUW5z3qM5): protocol: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection Aug 10 22:52:34.957729272 (53h26m29.388809393s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tvoTD1LoSj3q2rLiDckfT56dXNKKwdVu1ZAsBZExNFVTom9tH): protocol: rpc: dial tcp 104.36.82.8:4995: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957748227 (53h26m29.388828348s) [seg_pos={0 0}, upload=55724]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 22:52:34.957846646 (53h26m29.388926767s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 22:52:34.957851144 (53h26m29.388931275s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:52:35.306840333 (53h26m29.737920484s) [upload=55724]: Done uploading segments... Aug 10 22:52:35.306857145 (53h26m29.737937276s) [upload=55724]: Upload finished. err: <nil> Duration: 8.309694468s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 1.933543649s FAIL (2025-08-10 23:02:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 10 22:33:23.090759975 (53h7m17.521840126s) [upload=55388]: Starting upload Aug 10 22:33:23.090831261 (53h7m17.521911422s) [upload=55388]: Done starting upload Aug 10 22:33:23.090903269 (53h7m17.52198342s) [upload=55388]: Uploading segments... Aug 10 22:33:23.090955248 (53h7m17.52203541s) [upload=55388]: Got next segment. Inline: true Aug 10 22:33:23.090961811 (53h7m17.522041962s) [upload=55388]: Waiting for error group managing segments... Aug 10 22:33:23.091016636 (53h7m17.522096807s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:33:23.091023408 (53h7m17.52210356s) [upload=55388]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 22:33:23.091028138 (53h7m17.522108289s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:33:23.358287015 (53h7m17.789367146s) [upload=55388]: Done uploading segments... Aug 10 22:33:23.358303086 (53h7m17.789383207s) [upload=55388]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.933571062s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................XX...............................] duration: 6.991116909s FAIL (2025-08-10 23:01:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.991197814s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 8 out of 60 is failed (threshold=3) [X...............X.......X.X.....X.X...........XX............] duration: 6.921202721s FAIL (2025-08-10 23:01:55 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 22:52:30.226093181 (53h26m24.657173312s) [upload=55724]: Starting upload Aug 10 22:52:30.226146663 (53h26m24.657226794s) [upload=55724]: Done starting upload Aug 10 22:52:30.226228670 (53h26m24.657308801s) [upload=55724]: Uploading segments... Aug 10 22:52:30.226383958 (53h26m24.657464089s) [upload=55724]: Got next segment. Inline: false Aug 10 22:52:30.226415327 (53h26m24.657495458s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:52:30.226421208 (53h26m24.65750134s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 22:52:30.677132253 (53h26m25.108212384s) [seg_pos={0 0}, upload=55724]: Begin upload segment... Aug 10 22:52:30.677190715 (53h26m25.108270846s) [seg_pos={0 0}, upload=55724]: Done begin upload segment. Aug 10 22:52:30.677225091 (53h26m25.108305212s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677252474 (53h26m25.108332605s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677613114 (53h26m25.108693245s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678313253 (53h26m25.109393384s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678331708 (53h26m25.109411839s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678613808 (53h26m25.109693939s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678652442 (53h26m25.109732573s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678866331 (53h26m25.109946462s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679169000 (53h26m25.110249131s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679440199 (53h26m25.11052033s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679713732 (53h26m25.110793863s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679926259 (53h26m25.11100639s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680126402 (53h26m25.111206533s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680326895 (53h26m25.111407026s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680530465 (53h26m25.111610596s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680733313 (53h26m25.111813444s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681408626 (53h26m25.112488777s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681726623 (53h26m25.112806755s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:31.876216927 (53h26m26.307297058s) [upload=55724]: Next returned nil segment Aug 10 22:52:31.876232165 (53h26m26.307312296s) [upload=55724]: Waiting for error group managing segments... Aug 10 22:52:32.182969622 (53h26m26.614049743s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.631903297 (53h26m27.062983428s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.652429340 (53h26m27.083509471s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.695199823 (53h26m27.126279954s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.740711713 (53h26m27.171791844s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.776799260 (53h26m27.207879391s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.823959072 (53h26m27.255039203s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.873921155 (53h26m27.305001286s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886211631 (53h26m27.317291762s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886240236 (53h26m27.317320367s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:33.361231516 (53h26m27.792311647s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.095423782 (53h26m28.526503913s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.191766598 (53h26m28.622846729s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957234535 (53h26m29.388314666s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957270263 (53h26m29.388350384s) [seg_pos={0 0}, upload=55724]: Segment reached optimal threshold of 14 pieces. Aug 10 22:52:34.957519320 (53h26m29.388599441s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ezboNAWPbaK9UDENYoBb3iHjNU2ahn7mikiteN7y3Aj6uw4Nx): protocol: context canceled; context canceled Aug 10 22:52:34.957638617 (53h26m29.388718748s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SwpAaXTA3BqUvwubzmJkvwokdz6SSpT9t8HPtQcXXJxmcgfm2): protocol: read tcp 188.166.225.170:38236->104.36.82.37:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957696528 (53h26m29.388776659s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2euZtUdrPZGo2PieqDnKtSftgtfdTRk6V8VfYS4yUW5z3qM5): protocol: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection Aug 10 22:52:34.957729272 (53h26m29.388809393s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tvoTD1LoSj3q2rLiDckfT56dXNKKwdVu1ZAsBZExNFVTom9tH): protocol: rpc: dial tcp 104.36.82.8:4995: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957748227 (53h26m29.388828348s) [seg_pos={0 0}, upload=55724]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 22:52:34.957846646 (53h26m29.388926767s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 22:52:34.957851144 (53h26m29.388931275s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:52:35.306840333 (53h26m29.737920484s) [upload=55724]: Done uploading segments... Aug 10 22:52:35.306857145 (53h26m29.737937276s) [upload=55724]: Upload finished. err: <nil> Duration: 6.921233541s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 7.200311923s FAIL (2025-08-10 23:01:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 7.200338205s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................XX..............................] duration: 7.493345937s FAIL (2025-08-10 23:01:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 7.493376866s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.080384604s FAIL (2025-08-10 23:01:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 10 22:33:23.090759975 (53h7m17.521840126s) [upload=55388]: Starting upload Aug 10 22:33:23.090831261 (53h7m17.521911422s) [upload=55388]: Done starting upload Aug 10 22:33:23.090903269 (53h7m17.52198342s) [upload=55388]: Uploading segments... Aug 10 22:33:23.090955248 (53h7m17.52203541s) [upload=55388]: Got next segment. Inline: true Aug 10 22:33:23.090961811 (53h7m17.522041962s) [upload=55388]: Waiting for error group managing segments... Aug 10 22:33:23.091016636 (53h7m17.522096807s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:33:23.091023408 (53h7m17.52210356s) [upload=55388]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 22:33:23.091028138 (53h7m17.522108289s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:33:23.358287015 (53h7m17.789367146s) [upload=55388]: Done uploading segments... Aug 10 22:33:23.358303086 (53h7m17.789383207s) [upload=55388]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.080400253s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 8 out of 60 is failed (threshold=3) [.X...............X.......X.X.....X.X...........XX...........] duration: 7.957286808s FAIL (2025-08-10 23:01:02 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 22:52:30.226093181 (53h26m24.657173312s) [upload=55724]: Starting upload Aug 10 22:52:30.226146663 (53h26m24.657226794s) [upload=55724]: Done starting upload Aug 10 22:52:30.226228670 (53h26m24.657308801s) [upload=55724]: Uploading segments... Aug 10 22:52:30.226383958 (53h26m24.657464089s) [upload=55724]: Got next segment. Inline: false Aug 10 22:52:30.226415327 (53h26m24.657495458s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:52:30.226421208 (53h26m24.65750134s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 22:52:30.677132253 (53h26m25.108212384s) [seg_pos={0 0}, upload=55724]: Begin upload segment... Aug 10 22:52:30.677190715 (53h26m25.108270846s) [seg_pos={0 0}, upload=55724]: Done begin upload segment. Aug 10 22:52:30.677225091 (53h26m25.108305212s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677252474 (53h26m25.108332605s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677613114 (53h26m25.108693245s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678313253 (53h26m25.109393384s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678331708 (53h26m25.109411839s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678613808 (53h26m25.109693939s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678652442 (53h26m25.109732573s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678866331 (53h26m25.109946462s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679169000 (53h26m25.110249131s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679440199 (53h26m25.11052033s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679713732 (53h26m25.110793863s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679926259 (53h26m25.11100639s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680126402 (53h26m25.111206533s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680326895 (53h26m25.111407026s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680530465 (53h26m25.111610596s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680733313 (53h26m25.111813444s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681408626 (53h26m25.112488777s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681726623 (53h26m25.112806755s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:31.876216927 (53h26m26.307297058s) [upload=55724]: Next returned nil segment Aug 10 22:52:31.876232165 (53h26m26.307312296s) [upload=55724]: Waiting for error group managing segments... Aug 10 22:52:32.182969622 (53h26m26.614049743s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.631903297 (53h26m27.062983428s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.652429340 (53h26m27.083509471s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.695199823 (53h26m27.126279954s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.740711713 (53h26m27.171791844s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.776799260 (53h26m27.207879391s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.823959072 (53h26m27.255039203s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.873921155 (53h26m27.305001286s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886211631 (53h26m27.317291762s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886240236 (53h26m27.317320367s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:33.361231516 (53h26m27.792311647s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.095423782 (53h26m28.526503913s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.191766598 (53h26m28.622846729s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957234535 (53h26m29.388314666s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957270263 (53h26m29.388350384s) [seg_pos={0 0}, upload=55724]: Segment reached optimal threshold of 14 pieces. Aug 10 22:52:34.957519320 (53h26m29.388599441s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ezboNAWPbaK9UDENYoBb3iHjNU2ahn7mikiteN7y3Aj6uw4Nx): protocol: context canceled; context canceled Aug 10 22:52:34.957638617 (53h26m29.388718748s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SwpAaXTA3BqUvwubzmJkvwokdz6SSpT9t8HPtQcXXJxmcgfm2): protocol: read tcp 188.166.225.170:38236->104.36.82.37:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957696528 (53h26m29.388776659s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2euZtUdrPZGo2PieqDnKtSftgtfdTRk6V8VfYS4yUW5z3qM5): protocol: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection Aug 10 22:52:34.957729272 (53h26m29.388809393s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tvoTD1LoSj3q2rLiDckfT56dXNKKwdVu1ZAsBZExNFVTom9tH): protocol: rpc: dial tcp 104.36.82.8:4995: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957748227 (53h26m29.388828348s) [seg_pos={0 0}, upload=55724]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 22:52:34.957846646 (53h26m29.388926767s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 22:52:34.957851144 (53h26m29.388931275s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:52:35.306840333 (53h26m29.737920484s) [upload=55724]: Done uploading segments... Aug 10 22:52:35.306857145 (53h26m29.737937276s) [upload=55724]: Upload finished. err: <nil> Duration: 7.957316756s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 8 out of 60 is failed (threshold=3) [..X...............X.......X.X.....X.X...........XX..........] duration: 9.276873662s FAIL (2025-08-10 23:00:46 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 22:52:30.226093181 (53h26m24.657173312s) [upload=55724]: Starting upload Aug 10 22:52:30.226146663 (53h26m24.657226794s) [upload=55724]: Done starting upload Aug 10 22:52:30.226228670 (53h26m24.657308801s) [upload=55724]: Uploading segments... Aug 10 22:52:30.226383958 (53h26m24.657464089s) [upload=55724]: Got next segment. Inline: false Aug 10 22:52:30.226415327 (53h26m24.657495458s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:52:30.226421208 (53h26m24.65750134s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 22:52:30.677132253 (53h26m25.108212384s) [seg_pos={0 0}, upload=55724]: Begin upload segment... Aug 10 22:52:30.677190715 (53h26m25.108270846s) [seg_pos={0 0}, upload=55724]: Done begin upload segment. Aug 10 22:52:30.677225091 (53h26m25.108305212s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677252474 (53h26m25.108332605s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677613114 (53h26m25.108693245s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678313253 (53h26m25.109393384s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678331708 (53h26m25.109411839s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678613808 (53h26m25.109693939s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678652442 (53h26m25.109732573s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678866331 (53h26m25.109946462s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679169000 (53h26m25.110249131s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679440199 (53h26m25.11052033s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679713732 (53h26m25.110793863s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679926259 (53h26m25.11100639s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680126402 (53h26m25.111206533s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680326895 (53h26m25.111407026s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680530465 (53h26m25.111610596s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680733313 (53h26m25.111813444s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681408626 (53h26m25.112488777s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681726623 (53h26m25.112806755s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:31.876216927 (53h26m26.307297058s) [upload=55724]: Next returned nil segment Aug 10 22:52:31.876232165 (53h26m26.307312296s) [upload=55724]: Waiting for error group managing segments... Aug 10 22:52:32.182969622 (53h26m26.614049743s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.631903297 (53h26m27.062983428s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.652429340 (53h26m27.083509471s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.695199823 (53h26m27.126279954s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.740711713 (53h26m27.171791844s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.776799260 (53h26m27.207879391s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.823959072 (53h26m27.255039203s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.873921155 (53h26m27.305001286s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886211631 (53h26m27.317291762s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886240236 (53h26m27.317320367s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:33.361231516 (53h26m27.792311647s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.095423782 (53h26m28.526503913s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.191766598 (53h26m28.622846729s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957234535 (53h26m29.388314666s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957270263 (53h26m29.388350384s) [seg_pos={0 0}, upload=55724]: Segment reached optimal threshold of 14 pieces. Aug 10 22:52:34.957519320 (53h26m29.388599441s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ezboNAWPbaK9UDENYoBb3iHjNU2ahn7mikiteN7y3Aj6uw4Nx): protocol: context canceled; context canceled Aug 10 22:52:34.957638617 (53h26m29.388718748s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SwpAaXTA3BqUvwubzmJkvwokdz6SSpT9t8HPtQcXXJxmcgfm2): protocol: read tcp 188.166.225.170:38236->104.36.82.37:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957696528 (53h26m29.388776659s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2euZtUdrPZGo2PieqDnKtSftgtfdTRk6V8VfYS4yUW5z3qM5): protocol: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection Aug 10 22:52:34.957729272 (53h26m29.388809393s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tvoTD1LoSj3q2rLiDckfT56dXNKKwdVu1ZAsBZExNFVTom9tH): protocol: rpc: dial tcp 104.36.82.8:4995: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957748227 (53h26m29.388828348s) [seg_pos={0 0}, upload=55724]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 22:52:34.957846646 (53h26m29.388926767s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 22:52:34.957851144 (53h26m29.388931275s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:52:35.306840333 (53h26m29.737920484s) [upload=55724]: Done uploading segments... Aug 10 22:52:35.306857145 (53h26m29.737937276s) [upload=55724]: Upload finished. err: <nil> Duration: 9.276899912s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.808500561s FAIL (2025-08-10 23:00:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 6.808533666s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................XX.............................] duration: 7.362975417s FAIL (2025-08-10 23:00:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 7.363001737s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.601244339s FAIL (2025-08-10 23:00:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 6.601386863s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 9 out of 60 is failed (threshold=3) [X..X...............X.......X.X.....X.X...........XX.........] duration: 9.042717924s FAIL (2025-08-10 23:00:05 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 22:52:30.226093181 (53h26m24.657173312s) [upload=55724]: Starting upload Aug 10 22:52:30.226146663 (53h26m24.657226794s) [upload=55724]: Done starting upload Aug 10 22:52:30.226228670 (53h26m24.657308801s) [upload=55724]: Uploading segments... Aug 10 22:52:30.226383958 (53h26m24.657464089s) [upload=55724]: Got next segment. Inline: false Aug 10 22:52:30.226415327 (53h26m24.657495458s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:52:30.226421208 (53h26m24.65750134s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 22:52:30.677132253 (53h26m25.108212384s) [seg_pos={0 0}, upload=55724]: Begin upload segment... Aug 10 22:52:30.677190715 (53h26m25.108270846s) [seg_pos={0 0}, upload=55724]: Done begin upload segment. Aug 10 22:52:30.677225091 (53h26m25.108305212s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677252474 (53h26m25.108332605s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677613114 (53h26m25.108693245s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678313253 (53h26m25.109393384s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678331708 (53h26m25.109411839s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678613808 (53h26m25.109693939s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678652442 (53h26m25.109732573s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678866331 (53h26m25.109946462s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679169000 (53h26m25.110249131s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679440199 (53h26m25.11052033s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679713732 (53h26m25.110793863s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679926259 (53h26m25.11100639s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680126402 (53h26m25.111206533s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680326895 (53h26m25.111407026s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680530465 (53h26m25.111610596s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680733313 (53h26m25.111813444s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681408626 (53h26m25.112488777s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681726623 (53h26m25.112806755s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:31.876216927 (53h26m26.307297058s) [upload=55724]: Next returned nil segment Aug 10 22:52:31.876232165 (53h26m26.307312296s) [upload=55724]: Waiting for error group managing segments... Aug 10 22:52:32.182969622 (53h26m26.614049743s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.631903297 (53h26m27.062983428s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.652429340 (53h26m27.083509471s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.695199823 (53h26m27.126279954s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.740711713 (53h26m27.171791844s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.776799260 (53h26m27.207879391s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.823959072 (53h26m27.255039203s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.873921155 (53h26m27.305001286s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886211631 (53h26m27.317291762s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886240236 (53h26m27.317320367s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:33.361231516 (53h26m27.792311647s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.095423782 (53h26m28.526503913s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.191766598 (53h26m28.622846729s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957234535 (53h26m29.388314666s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957270263 (53h26m29.388350384s) [seg_pos={0 0}, upload=55724]: Segment reached optimal threshold of 14 pieces. Aug 10 22:52:34.957519320 (53h26m29.388599441s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ezboNAWPbaK9UDENYoBb3iHjNU2ahn7mikiteN7y3Aj6uw4Nx): protocol: context canceled; context canceled Aug 10 22:52:34.957638617 (53h26m29.388718748s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SwpAaXTA3BqUvwubzmJkvwokdz6SSpT9t8HPtQcXXJxmcgfm2): protocol: read tcp 188.166.225.170:38236->104.36.82.37:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957696528 (53h26m29.388776659s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2euZtUdrPZGo2PieqDnKtSftgtfdTRk6V8VfYS4yUW5z3qM5): protocol: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection Aug 10 22:52:34.957729272 (53h26m29.388809393s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tvoTD1LoSj3q2rLiDckfT56dXNKKwdVu1ZAsBZExNFVTom9tH): protocol: rpc: dial tcp 104.36.82.8:4995: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957748227 (53h26m29.388828348s) [seg_pos={0 0}, upload=55724]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 22:52:34.957846646 (53h26m29.388926767s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 22:52:34.957851144 (53h26m29.388931275s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:52:35.306840333 (53h26m29.737920484s) [upload=55724]: Done uploading segments... Aug 10 22:52:35.306857145 (53h26m29.737937276s) [upload=55724]: Upload finished. err: <nil> Duration: 9.042749504s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 1.968653315s FAIL (2025-08-10 23:00:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 10 22:33:23.090759975 (53h7m17.521840126s) [upload=55388]: Starting upload Aug 10 22:33:23.090831261 (53h7m17.521911422s) [upload=55388]: Done starting upload Aug 10 22:33:23.090903269 (53h7m17.52198342s) [upload=55388]: Uploading segments... Aug 10 22:33:23.090955248 (53h7m17.52203541s) [upload=55388]: Got next segment. Inline: true Aug 10 22:33:23.090961811 (53h7m17.522041962s) [upload=55388]: Waiting for error group managing segments... Aug 10 22:33:23.091016636 (53h7m17.522096807s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:33:23.091023408 (53h7m17.52210356s) [upload=55388]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 22:33:23.091028138 (53h7m17.522108289s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:33:23.358287015 (53h7m17.789367146s) [upload=55388]: Done uploading segments... Aug 10 22:33:23.358303086 (53h7m17.789383207s) [upload=55388]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.968690516s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 7.280944585s FAIL (2025-08-10 22:59:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 7.280970233s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................XX............................] duration: 7.45128957s FAIL (2025-08-10 22:59:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 7.451316149s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.179145018s FAIL (2025-08-10 22:59:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 10 22:33:23.090759975 (53h7m17.521840126s) [upload=55388]: Starting upload Aug 10 22:33:23.090831261 (53h7m17.521911422s) [upload=55388]: Done starting upload Aug 10 22:33:23.090903269 (53h7m17.52198342s) [upload=55388]: Uploading segments... Aug 10 22:33:23.090955248 (53h7m17.52203541s) [upload=55388]: Got next segment. Inline: true Aug 10 22:33:23.090961811 (53h7m17.522041962s) [upload=55388]: Waiting for error group managing segments... Aug 10 22:33:23.091016636 (53h7m17.522096807s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:33:23.091023408 (53h7m17.52210356s) [upload=55388]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 22:33:23.091028138 (53h7m17.522108289s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:33:23.358287015 (53h7m17.789367146s) [upload=55388]: Done uploading segments... Aug 10 22:33:23.358303086 (53h7m17.789383207s) [upload=55388]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.179170767s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 9 out of 60 is failed (threshold=3) [.X..X...............X.......X.X.....X.X...........XX........] duration: 9.171350152s FAIL (2025-08-10 22:59:06 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 22:52:30.226093181 (53h26m24.657173312s) [upload=55724]: Starting upload Aug 10 22:52:30.226146663 (53h26m24.657226794s) [upload=55724]: Done starting upload Aug 10 22:52:30.226228670 (53h26m24.657308801s) [upload=55724]: Uploading segments... Aug 10 22:52:30.226383958 (53h26m24.657464089s) [upload=55724]: Got next segment. Inline: false Aug 10 22:52:30.226415327 (53h26m24.657495458s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:52:30.226421208 (53h26m24.65750134s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 22:52:30.677132253 (53h26m25.108212384s) [seg_pos={0 0}, upload=55724]: Begin upload segment... Aug 10 22:52:30.677190715 (53h26m25.108270846s) [seg_pos={0 0}, upload=55724]: Done begin upload segment. Aug 10 22:52:30.677225091 (53h26m25.108305212s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677252474 (53h26m25.108332605s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677613114 (53h26m25.108693245s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678313253 (53h26m25.109393384s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678331708 (53h26m25.109411839s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678613808 (53h26m25.109693939s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678652442 (53h26m25.109732573s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678866331 (53h26m25.109946462s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679169000 (53h26m25.110249131s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679440199 (53h26m25.11052033s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679713732 (53h26m25.110793863s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679926259 (53h26m25.11100639s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680126402 (53h26m25.111206533s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680326895 (53h26m25.111407026s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680530465 (53h26m25.111610596s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680733313 (53h26m25.111813444s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681408626 (53h26m25.112488777s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681726623 (53h26m25.112806755s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:31.876216927 (53h26m26.307297058s) [upload=55724]: Next returned nil segment Aug 10 22:52:31.876232165 (53h26m26.307312296s) [upload=55724]: Waiting for error group managing segments... Aug 10 22:52:32.182969622 (53h26m26.614049743s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.631903297 (53h26m27.062983428s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.652429340 (53h26m27.083509471s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.695199823 (53h26m27.126279954s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.740711713 (53h26m27.171791844s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.776799260 (53h26m27.207879391s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.823959072 (53h26m27.255039203s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.873921155 (53h26m27.305001286s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886211631 (53h26m27.317291762s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886240236 (53h26m27.317320367s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:33.361231516 (53h26m27.792311647s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.095423782 (53h26m28.526503913s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.191766598 (53h26m28.622846729s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957234535 (53h26m29.388314666s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957270263 (53h26m29.388350384s) [seg_pos={0 0}, upload=55724]: Segment reached optimal threshold of 14 pieces. Aug 10 22:52:34.957519320 (53h26m29.388599441s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ezboNAWPbaK9UDENYoBb3iHjNU2ahn7mikiteN7y3Aj6uw4Nx): protocol: context canceled; context canceled Aug 10 22:52:34.957638617 (53h26m29.388718748s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SwpAaXTA3BqUvwubzmJkvwokdz6SSpT9t8HPtQcXXJxmcgfm2): protocol: read tcp 188.166.225.170:38236->104.36.82.37:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957696528 (53h26m29.388776659s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2euZtUdrPZGo2PieqDnKtSftgtfdTRk6V8VfYS4yUW5z3qM5): protocol: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection Aug 10 22:52:34.957729272 (53h26m29.388809393s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tvoTD1LoSj3q2rLiDckfT56dXNKKwdVu1ZAsBZExNFVTom9tH): protocol: rpc: dial tcp 104.36.82.8:4995: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957748227 (53h26m29.388828348s) [seg_pos={0 0}, upload=55724]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 22:52:34.957846646 (53h26m29.388926767s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 22:52:34.957851144 (53h26m29.388931275s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:52:35.306840333 (53h26m29.737920484s) [upload=55724]: Done uploading segments... Aug 10 22:52:35.306857145 (53h26m29.737937276s) [upload=55724]: Upload finished. err: <nil> Duration: 9.171374088s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 1.87765597s FAIL (2025-08-10 22:59:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 10 22:33:23.090759975 (53h7m17.521840126s) [upload=55388]: Starting upload Aug 10 22:33:23.090831261 (53h7m17.521911422s) [upload=55388]: Done starting upload Aug 10 22:33:23.090903269 (53h7m17.52198342s) [upload=55388]: Uploading segments... Aug 10 22:33:23.090955248 (53h7m17.52203541s) [upload=55388]: Got next segment. Inline: true Aug 10 22:33:23.090961811 (53h7m17.522041962s) [upload=55388]: Waiting for error group managing segments... Aug 10 22:33:23.091016636 (53h7m17.522096807s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:33:23.091023408 (53h7m17.52210356s) [upload=55388]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 22:33:23.091028138 (53h7m17.522108289s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:33:23.358287015 (53h7m17.789367146s) [upload=55388]: Done uploading segments... Aug 10 22:33:23.358303086 (53h7m17.789383207s) [upload=55388]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.877673423s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................XX...........................] duration: 6.654253671s FAIL (2025-08-10 22:59:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.654282326s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.594205376s FAIL (2025-08-10 22:58:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 6.59423317s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................XX..........................] duration: 7.017231453s FAIL (2025-08-10 22:58:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 7.017265597s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.399530069s FAIL (2025-08-10 22:58:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 6.399549075s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 1.908611127s FAIL (2025-08-10 22:58:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 10 22:33:23.090759975 (53h7m17.521840126s) [upload=55388]: Starting upload Aug 10 22:33:23.090831261 (53h7m17.521911422s) [upload=55388]: Done starting upload Aug 10 22:33:23.090903269 (53h7m17.52198342s) [upload=55388]: Uploading segments... Aug 10 22:33:23.090955248 (53h7m17.52203541s) [upload=55388]: Got next segment. Inline: true Aug 10 22:33:23.090961811 (53h7m17.522041962s) [upload=55388]: Waiting for error group managing segments... Aug 10 22:33:23.091016636 (53h7m17.522096807s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:33:23.091023408 (53h7m17.52210356s) [upload=55388]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 22:33:23.091028138 (53h7m17.522108289s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:33:23.358287015 (53h7m17.789367146s) [upload=55388]: Done uploading segments... Aug 10 22:33:23.358303086 (53h7m17.789383207s) [upload=55388]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.908641225s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 9 out of 60 is failed (threshold=3) [..X..X...............X.......X.X.....X.X...........XX.......] duration: 6.760084134s FAIL (2025-08-10 22:57:54 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 22:52:30.226093181 (53h26m24.657173312s) [upload=55724]: Starting upload Aug 10 22:52:30.226146663 (53h26m24.657226794s) [upload=55724]: Done starting upload Aug 10 22:52:30.226228670 (53h26m24.657308801s) [upload=55724]: Uploading segments... Aug 10 22:52:30.226383958 (53h26m24.657464089s) [upload=55724]: Got next segment. Inline: false Aug 10 22:52:30.226415327 (53h26m24.657495458s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:52:30.226421208 (53h26m24.65750134s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 22:52:30.677132253 (53h26m25.108212384s) [seg_pos={0 0}, upload=55724]: Begin upload segment... Aug 10 22:52:30.677190715 (53h26m25.108270846s) [seg_pos={0 0}, upload=55724]: Done begin upload segment. Aug 10 22:52:30.677225091 (53h26m25.108305212s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677252474 (53h26m25.108332605s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677613114 (53h26m25.108693245s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678313253 (53h26m25.109393384s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678331708 (53h26m25.109411839s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678613808 (53h26m25.109693939s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678652442 (53h26m25.109732573s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678866331 (53h26m25.109946462s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679169000 (53h26m25.110249131s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679440199 (53h26m25.11052033s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679713732 (53h26m25.110793863s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679926259 (53h26m25.11100639s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680126402 (53h26m25.111206533s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680326895 (53h26m25.111407026s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680530465 (53h26m25.111610596s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680733313 (53h26m25.111813444s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681408626 (53h26m25.112488777s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681726623 (53h26m25.112806755s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:31.876216927 (53h26m26.307297058s) [upload=55724]: Next returned nil segment Aug 10 22:52:31.876232165 (53h26m26.307312296s) [upload=55724]: Waiting for error group managing segments... Aug 10 22:52:32.182969622 (53h26m26.614049743s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.631903297 (53h26m27.062983428s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.652429340 (53h26m27.083509471s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.695199823 (53h26m27.126279954s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.740711713 (53h26m27.171791844s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.776799260 (53h26m27.207879391s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.823959072 (53h26m27.255039203s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.873921155 (53h26m27.305001286s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886211631 (53h26m27.317291762s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886240236 (53h26m27.317320367s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:33.361231516 (53h26m27.792311647s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.095423782 (53h26m28.526503913s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.191766598 (53h26m28.622846729s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957234535 (53h26m29.388314666s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957270263 (53h26m29.388350384s) [seg_pos={0 0}, upload=55724]: Segment reached optimal threshold of 14 pieces. Aug 10 22:52:34.957519320 (53h26m29.388599441s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ezboNAWPbaK9UDENYoBb3iHjNU2ahn7mikiteN7y3Aj6uw4Nx): protocol: context canceled; context canceled Aug 10 22:52:34.957638617 (53h26m29.388718748s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SwpAaXTA3BqUvwubzmJkvwokdz6SSpT9t8HPtQcXXJxmcgfm2): protocol: read tcp 188.166.225.170:38236->104.36.82.37:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957696528 (53h26m29.388776659s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2euZtUdrPZGo2PieqDnKtSftgtfdTRk6V8VfYS4yUW5z3qM5): protocol: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection Aug 10 22:52:34.957729272 (53h26m29.388809393s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tvoTD1LoSj3q2rLiDckfT56dXNKKwdVu1ZAsBZExNFVTom9tH): protocol: rpc: dial tcp 104.36.82.8:4995: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957748227 (53h26m29.388828348s) [seg_pos={0 0}, upload=55724]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 22:52:34.957846646 (53h26m29.388926767s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 22:52:34.957851144 (53h26m29.388931275s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:52:35.306840333 (53h26m29.737920484s) [upload=55724]: Done uploading segments... Aug 10 22:52:35.306857145 (53h26m29.737937276s) [upload=55724]: Upload finished. err: <nil> Duration: 6.760109462s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................XX.........................] duration: 10.081475296s FAIL (2025-08-10 22:57:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 10.081506146s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 6.140517402s FAIL (2025-08-10 22:57:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 6.140539754s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 9 out of 60 is failed (threshold=3) [...X..X...............X.......X.X.....X.X...........XX......] duration: 7.259411314s FAIL (2025-08-10 22:57:37 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 22:52:30.226093181 (53h26m24.657173312s) [upload=55724]: Starting upload Aug 10 22:52:30.226146663 (53h26m24.657226794s) [upload=55724]: Done starting upload Aug 10 22:52:30.226228670 (53h26m24.657308801s) [upload=55724]: Uploading segments... Aug 10 22:52:30.226383958 (53h26m24.657464089s) [upload=55724]: Got next segment. Inline: false Aug 10 22:52:30.226415327 (53h26m24.657495458s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:52:30.226421208 (53h26m24.65750134s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 22:52:30.677132253 (53h26m25.108212384s) [seg_pos={0 0}, upload=55724]: Begin upload segment... Aug 10 22:52:30.677190715 (53h26m25.108270846s) [seg_pos={0 0}, upload=55724]: Done begin upload segment. Aug 10 22:52:30.677225091 (53h26m25.108305212s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677252474 (53h26m25.108332605s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677613114 (53h26m25.108693245s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678313253 (53h26m25.109393384s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678331708 (53h26m25.109411839s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678613808 (53h26m25.109693939s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678652442 (53h26m25.109732573s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678866331 (53h26m25.109946462s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679169000 (53h26m25.110249131s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679440199 (53h26m25.11052033s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679713732 (53h26m25.110793863s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679926259 (53h26m25.11100639s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680126402 (53h26m25.111206533s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680326895 (53h26m25.111407026s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680530465 (53h26m25.111610596s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680733313 (53h26m25.111813444s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681408626 (53h26m25.112488777s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681726623 (53h26m25.112806755s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:31.876216927 (53h26m26.307297058s) [upload=55724]: Next returned nil segment Aug 10 22:52:31.876232165 (53h26m26.307312296s) [upload=55724]: Waiting for error group managing segments... Aug 10 22:52:32.182969622 (53h26m26.614049743s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.631903297 (53h26m27.062983428s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.652429340 (53h26m27.083509471s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.695199823 (53h26m27.126279954s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.740711713 (53h26m27.171791844s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.776799260 (53h26m27.207879391s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.823959072 (53h26m27.255039203s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.873921155 (53h26m27.305001286s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886211631 (53h26m27.317291762s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886240236 (53h26m27.317320367s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:33.361231516 (53h26m27.792311647s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.095423782 (53h26m28.526503913s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.191766598 (53h26m28.622846729s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957234535 (53h26m29.388314666s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957270263 (53h26m29.388350384s) [seg_pos={0 0}, upload=55724]: Segment reached optimal threshold of 14 pieces. Aug 10 22:52:34.957519320 (53h26m29.388599441s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ezboNAWPbaK9UDENYoBb3iHjNU2ahn7mikiteN7y3Aj6uw4Nx): protocol: context canceled; context canceled Aug 10 22:52:34.957638617 (53h26m29.388718748s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SwpAaXTA3BqUvwubzmJkvwokdz6SSpT9t8HPtQcXXJxmcgfm2): protocol: read tcp 188.166.225.170:38236->104.36.82.37:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957696528 (53h26m29.388776659s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2euZtUdrPZGo2PieqDnKtSftgtfdTRk6V8VfYS4yUW5z3qM5): protocol: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection Aug 10 22:52:34.957729272 (53h26m29.388809393s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tvoTD1LoSj3q2rLiDckfT56dXNKKwdVu1ZAsBZExNFVTom9tH): protocol: rpc: dial tcp 104.36.82.8:4995: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957748227 (53h26m29.388828348s) [seg_pos={0 0}, upload=55724]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 22:52:34.957846646 (53h26m29.388926767s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 22:52:34.957851144 (53h26m29.388931275s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:52:35.306840333 (53h26m29.737920484s) [upload=55724]: Done uploading segments... Aug 10 22:52:35.306857145 (53h26m29.737937276s) [upload=55724]: Upload finished. err: <nil> Duration: 7.259440441s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 1.886531571s FAIL (2025-08-10 22:57:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 10 22:33:23.090759975 (53h7m17.521840126s) [upload=55388]: Starting upload Aug 10 22:33:23.090831261 (53h7m17.521911422s) [upload=55388]: Done starting upload Aug 10 22:33:23.090903269 (53h7m17.52198342s) [upload=55388]: Uploading segments... Aug 10 22:33:23.090955248 (53h7m17.52203541s) [upload=55388]: Got next segment. Inline: true Aug 10 22:33:23.090961811 (53h7m17.522041962s) [upload=55388]: Waiting for error group managing segments... Aug 10 22:33:23.091016636 (53h7m17.522096807s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:33:23.091023408 (53h7m17.52210356s) [upload=55388]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 22:33:23.091028138 (53h7m17.522108289s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:33:23.358287015 (53h7m17.789367146s) [upload=55388]: Done uploading segments... Aug 10 22:33:23.358303086 (53h7m17.789383207s) [upload=55388]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.886556059s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................XX........................] duration: 7.832309113s FAIL (2025-08-10 22:57:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 7.832328981s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.245475986s FAIL (2025-08-10 22:57:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 6.24550387s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................XX.......................] duration: 6.211406535s FAIL (2025-08-10 22:57:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.211438396s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 2.121219153s FAIL (2025-08-10 22:57:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 10 22:33:23.090759975 (53h7m17.521840126s) [upload=55388]: Starting upload Aug 10 22:33:23.090831261 (53h7m17.521911422s) [upload=55388]: Done starting upload Aug 10 22:33:23.090903269 (53h7m17.52198342s) [upload=55388]: Uploading segments... Aug 10 22:33:23.090955248 (53h7m17.52203541s) [upload=55388]: Got next segment. Inline: true Aug 10 22:33:23.090961811 (53h7m17.522041962s) [upload=55388]: Waiting for error group managing segments... Aug 10 22:33:23.091016636 (53h7m17.522096807s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:33:23.091023408 (53h7m17.52210356s) [upload=55388]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 22:33:23.091028138 (53h7m17.522108289s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:33:23.358287015 (53h7m17.789367146s) [upload=55388]: Done uploading segments... Aug 10 22:33:23.358303086 (53h7m17.789383207s) [upload=55388]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.121257466s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................XX......................] duration: 9.96178131s FAIL (2025-08-10 22:56:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== loading piece... Aug 10 22:40:53.943549380 (53h14m48.374629521s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943556974 (53h14m48.374637095s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943597543 (53h14m48.374677674s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.943973151 (53h14m48.375053282s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944299355 (53h14m48.375379496s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944384608 (53h14m48.375464739s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944631009 (53h14m48.37571114s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944727565 (53h14m48.375807696s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.944860439 (53h14m48.37594057s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945088966 (53h14m48.376169097s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945147738 (53h14m48.376227869s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945353152 (53h14m48.376433283s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945606616 (53h14m48.376686748s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.945871724 (53h14m48.376951855s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.947941243 (53h14m48.379021374s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:53.948327823 (53h14m48.379407954s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Uploading piece... Aug 10 22:40:55.065322831 (53h14m49.496402962s) [upload=55525]: Next returned nil segment Aug 10 22:40:55.065342348 (53h14m49.49642251s) [upload=55525]: Waiting for error group managing segments... Aug 10 22:40:55.351936440 (53h14m49.783016571s) [noise=true , address=64.49.13.138:4488 , piece_id=RCCTYN7Y2OZIYWEKPURPYO4BSOLDHVHTCO63Y3BH5N46CN7CNXIA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.396100421 (53h14m49.827180552s) [noise=true , address=64.49.13.139:4470 , piece_id=DLZLLGII6NXMCWTFWGS3OF7LPULMLR4JPQMDKK6GCUN2DAMX625A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.406362685 (53h14m49.837442826s) [noise=true , address=193.149.164.143:4701 , piece_id=QHI2XVOX7W2UGF4BQSLSD2VIVV52LLABIQFT3BXLZEUNG3ZLPIEA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.419436730 (53h14m49.850516861s) [noise=true , address=79.127.205.245:4240 , piece_id=XF4MLNG6VPY2WDQBQHEKZQA64TDPTL6WSAI6ZW6XWPEQC4ODBPWA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:40:55.430290006 (53h14m49.861370137s) [noise=true , address=79.127.205.234:4598 , piece_id=2EI6HK3UTDZ5XOY34OVQG3EKCD7SFPICWTSJHH4DQJBFOSETL5RQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:17.485741365 (53h15m11.916821486s) [noise=true , address=104.36.82.46:4498 , piece_id=BOR2M6TCV2A5NHT4X62SF66ZMQUE2YB2WBNH56ANZTE367RACQXA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:24.084063242 (53h15m18.515143373s) [noise=true , address=104.36.82.23:4452 , piece_id=P4PEW5G2HXNZSASQCYCWYTQFYI6XG63A5INLBHVOOCE5HH4BPRHQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:28.527843834 (53h15m22.958923965s) [noise=true , address=104.36.82.47:4541 , piece_id=LC5QTRQGGXQ6CG4VKGGLKPFLBPOZ46I5IUN5EEVRZNBXCLIV4CCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:33.874254204 (53h15m28.305334335s) [noise=true , address=104.36.82.37:4677 , piece_id=7FGAV3NNV7GBZK5MNEI35SXTGL2JZ7FHKYOHFSSFPAOCZJ5UI75Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:47.512694556 (53h15m41.943774687s) [noise=true , address=104.36.82.38:4722 , piece_id=SMBUNZENTKSSKSSYLKBRD6QA4FVMYFA655OBQXNW4FWYUQELTNCA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: <nil> Aug 10 22:41:52.846710159 (53h15m47.27779029s) [noise=true , address=104.36.82.45:4911 , piece_id=S4CVGLLIK3C62VB4WQKD6SER3AL3RVG235HXO2DD7VSFFKVI6IRQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12mzM4hseStJYeWqqm29qGjdaKkxydqFsKaKwNFrywGpDWSwFDQ, address:104.36.82.45:4911): protocol: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60468->104.36.82.45:4911: use of closed network connection Aug 10 22:41:52.846755196 (53h15m47.277835327s) [noise=true , address=104.36.82.35:4612 , piece_id=HIICHFXV4DN3VWYY4VS6CGRPDV2XIX25DHLJVSR3PIPUSPQRCWVA, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12jM3gpXbW8JvCB5bB2z3FoFmTEfZ9nsLeonzKzYmFT4SjyXUZV, address:104.36.82.35:4612): protocol: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46610->104.36.82.35:4612: use of closed network connection Aug 10 22:41:52.846783941 (53h15m47.277864072s) [noise=true , address=104.36.82.60:4339 , piece_id=4TQXIBW4F65FX25ZHDGVYO7ZJIZKJYZR6M5F32L6MBLMGAUVQIOQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:104.36.82.60:4339): protocol: rpc: dial tcp 104.36.82.60:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846820601 (53h15m47.277900722s) [noise=true , address=104.36.82.8:4977 , piece_id=GLYGCEHBCWPC2UGZKCPXVOM2BI34I2DL552Z7YZCLZ5YODKDIXBQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12WCAm12tXwimFSPqz7Seh6A4oxWqQVxqsF459xux4a5CqjpbAT, address:104.36.82.8:4977): protocol: rpc: dial tcp 104.36.82.8:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846853675 (53h15m47.277933806s) [noise=true , address=104.36.82.58:4279 , piece_id=PF6K7Q7GMBN237YTKOCSKUCSIVG4WZHAB5VM7XNQ7SZEVVJY2D4A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12LtRviHk2GinJGa3L53MCHCAfKrqUsey5JhQRHbYQTuSUNKoNC, address:104.36.82.58:4279): protocol: rpc: dial tcp 104.36.82.58:4279: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846860007 (53h15m47.277940138s) [noise=true , address=104.36.82.62:4430 , piece_id=U63SQHNDHSNPB3Y3OC4AOSPEAIKD5TI6COFPP7TXPCCSSIQM3WKQ, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:15fmq7zLwrSHRmwJ4eai2bXDnCsAqUEhqCLHqHvMuf189v9sQb, address:104.36.82.62:4430): protocol: write tcp 188.166.225.170:53320->104.36.82.62:4430: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846875696 (53h15m47.277955817s) [noise=true , address=104.36.82.10:4104 , piece_id=XA3M5X4ZYV75QK5M3OQL55HDTF6CYS5ZNOMUC5HNH3HNXSZDPF7Q, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:12th1zHy3xbqtrsUMY78iYg8WhCoubrbsuQXtLBcAC8hjEfo1NH, address:104.36.82.10:4104): protocol: rpc: dial tcp 104.36.82.10:4104: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846892138 (53h15m47.277972269s) [noise=true , address=104.36.82.18:4372 , piece_id=23XSYB4TPDQFFLVPRHHDOCA4AOAYJMQZMFPDT7EQHWHMPU6KT23A, seg_pos={0 0}, upload=55525]: Done uploading piece. err: ecclient: upload failed (node:1fchsL2RkNzQhjmtJgKigZUHhrVgb48PohuJNFk8GmKiVax44T, address:104.36.82.18:4372): protocol: rpc: dial tcp 104.36.82.18:4372: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:41:52.846913489 (53h15m47.27799362s) [seg_pos={0 0}, upload=55525]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 10 22:41:53.081503289 (53h15m47.51258341s) [upload=55525]: Done uploading segments... Aug 10 22:41:53.081530240 (53h15m47.512610362s) [upload=55525]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 9.961802311s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.610894847s FAIL (2025-08-10 22:56:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== ading piece... Aug 10 22:39:50.336074364 (53h13m44.767154495s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Uploading piece... Aug 10 22:40:19.456819087 (53h14m13.887899229s) [upload=55509]: Next segment err: context deadline exceeded Aug 10 22:40:19.457783993 (53h14m13.888864135s) [noise=true , address=45.91.116.5:4298 , piece_id=GVMGOZCJWYS7J3T6DHGZE2FV6HL765AH2GKAUA6YC4WO5VNYHK5Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38014->45.91.116.5:4298: use of closed network connection Aug 10 22:40:19.461082815 (53h14m13.892162946s) [noise=true , address=45.91.116.11:4119 , piece_id=GKGBSCCL2OE2QOW64DNOYLI6B3DRY42DRZFSYPHQPW4ZZJI3WZMQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34580->45.91.116.11:4119: use of closed network connection Aug 10 22:40:19.461163860 (53h14m13.892243991s) [noise=true , address=77.90.31.10:4519 , piece_id=54QOA6EMWRSZQ6Y2OJLISSTZZHXQ65BMVTPKSFSXTN7FX6GYSP4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53246->77.90.31.10:4519: use of closed network connection Aug 10 22:40:19.461232402 (53h14m13.892312533s) [noise=true , address=104.36.82.14:4222 , piece_id=YTNRW3RFBMSRNON7BINBR6CQ4RJ4RAUB22SRHKICXJHSB2RBTANQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461288870 (53h14m13.892369011s) [noise=true , address=104.36.82.8:4970 , piece_id=MK6SEEA5GOO633T5M3PZEBAI3SLIXHBJ6K6OC3VS26AUNF3BODCQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: rpc: dial tcp 104.36.82.8:4970: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:40:19.461349325 (53h14m13.892429466s) [noise=true , address=45.91.116.14:4190 , piece_id=WV3CT6AZVKYWCVFRX4A64OWDA3MDARBNZXZVAUQEPITZDHEJM4NQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41494->45.91.116.14:4190: use of closed network connection Aug 10 22:40:19.461405783 (53h14m13.892485914s) [noise=true , address=77.90.31.13:4627 , piece_id=FDUYEH66CG5N75FFPZO33VLDNXERXVRU777HRENJQ6AO3CE3PGXA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60098->77.90.31.13:4627: use of closed network connection Aug 10 22:40:19.461459857 (53h14m13.892539988s) [noise=true , address=45.91.116.4:4940 , piece_id=EGD74UMVIBGMBYFTATXD5MP736KKHKG3S4Z6XM6YIVRBQMS7WNFA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42802->45.91.116.4:4940: use of closed network connection Aug 10 22:40:19.461515543 (53h14m13.892595674s) [noise=true , address=77.90.31.17:4770 , piece_id=UCGDKAZLEO5NCHXBXPV6WS45GX4DQ6XMZV5OVOTH3YPZ76IGKMTQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43000->77.90.31.17:4770: use of closed network connection Aug 10 22:40:19.461591038 (53h14m13.892671169s) [noise=true , address=45.91.116.12:4225 , piece_id=EG2YTLPB7OYMJALBKBYNX4Y7U4WZO35MYSMRYI7YW3EQHYPTCLVA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54008->45.91.116.12:4225: use of closed network connection Aug 10 22:40:19.461652836 (53h14m13.892732967s) [noise=true , address=77.90.31.19:4841 , piece_id=FAYN4ZUGPDD2PLZOF3Z75UMYDIKQVWBFLDZ66IMEJS3PGZHLIN6A, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33564->77.90.31.19:4841: use of closed network connection Aug 10 22:40:19.461732308 (53h14m13.892812449s) [noise=true , address=77.90.31.12:4592 , piece_id=I47565EJHMGJPCSUXVXEWRZDHDSAJJAGCPCO7XTPLTIGY3IQW2ZQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42676->77.90.31.12:4592: use of closed network connection Aug 10 22:40:19.461887956 (53h14m13.892968087s) [noise=true , address=77.90.31.12:4590 , piece_id=XMSOJVMRJHDL7LL7FEJIBMQWXYN6J52B6GBLV6LV4XZIDHR47XYA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58782->77.90.31.12:4590: use of closed network connection Aug 10 22:40:19.461959753 (53h14m13.893039874s) [noise=true , address=45.91.116.10:4083 , piece_id=X4JN6C2QK55BWAUI7R4PMROM5VI5K3IZQBVLYLEWKVPDBQZHKB4Q, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46198->45.91.116.10:4083: use of closed network connection Aug 10 22:40:19.462013736 (53h14m13.893093867s) [noise=true , address=77.90.31.11:4554 , piece_id=UNJ3KNUZDBS7FW67TI7FHKVOVX4SQCLT5NEKRPOM4JS3BJI73CNQ, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36572->77.90.31.11:4554: use of closed network connection Aug 10 22:40:19.462072048 (53h14m13.893152179s) [noise=true , address=45.91.116.7:4009 , piece_id=6XPXA2QEPR7YLCVC3VED6BO5YMJ3QDZQ66QO5O3ZQUTOPAF7I6AA, seg_pos={0 0}, upload=55509]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50298->45.91.116.7:4009: use of closed network connection Aug 10 22:40:19.462087708 (53h14m13.893167839s) [seg_pos={0 0}, upload=55509]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 6.610930254s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 9 out of 60 is failed (threshold=3) [....X..X...............X.......X.X.....X.X...........XX.....] duration: 7.234735574s FAIL (2025-08-10 22:56:24 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 10 22:52:30.226093181 (53h26m24.657173312s) [upload=55724]: Starting upload Aug 10 22:52:30.226146663 (53h26m24.657226794s) [upload=55724]: Done starting upload Aug 10 22:52:30.226228670 (53h26m24.657308801s) [upload=55724]: Uploading segments... Aug 10 22:52:30.226383958 (53h26m24.657464089s) [upload=55724]: Got next segment. Inline: false Aug 10 22:52:30.226415327 (53h26m24.657495458s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:52:30.226421208 (53h26m24.65750134s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 10 22:52:30.677132253 (53h26m25.108212384s) [seg_pos={0 0}, upload=55724]: Begin upload segment... Aug 10 22:52:30.677190715 (53h26m25.108270846s) [seg_pos={0 0}, upload=55724]: Done begin upload segment. Aug 10 22:52:30.677225091 (53h26m25.108305212s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677252474 (53h26m25.108332605s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.677613114 (53h26m25.108693245s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678313253 (53h26m25.109393384s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678331708 (53h26m25.109411839s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678613808 (53h26m25.109693939s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678652442 (53h26m25.109732573s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.678866331 (53h26m25.109946462s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679169000 (53h26m25.110249131s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679440199 (53h26m25.11052033s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679713732 (53h26m25.110793863s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.679926259 (53h26m25.11100639s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680126402 (53h26m25.111206533s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680326895 (53h26m25.111407026s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680530465 (53h26m25.111610596s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.680733313 (53h26m25.111813444s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681408626 (53h26m25.112488777s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:30.681726623 (53h26m25.112806755s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Uploading piece... Aug 10 22:52:31.876216927 (53h26m26.307297058s) [upload=55724]: Next returned nil segment Aug 10 22:52:31.876232165 (53h26m26.307312296s) [upload=55724]: Waiting for error group managing segments... Aug 10 22:52:32.182969622 (53h26m26.614049743s) [noise=true , address=64.49.13.169:4713 , piece_id=7NZUZRKMLSKQ2YCASM264IOLWXRM5ZLQYAP53HL5BD52BLBCAEPA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.631903297 (53h26m27.062983428s) [noise=true , address=104.36.82.46:4507 , piece_id=B4NHEAVESRXDYUIOJ7A4WKZJOK4NPFT2JADD3JZXBPDVULAUEPIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.652429340 (53h26m27.083509471s) [noise=true , address=104.36.82.34:4564 , piece_id=AVAXGPPYXYSSZIIG47NV4RISUD4RPP7S6KTYF4QYSAPDM3NYOBDA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.695199823 (53h26m27.126279954s) [noise=true , address=104.36.82.16:4310 , piece_id=OXZLVTO5EASBCCHBSGFSHQSF2AI2U2XI6ENEIZPWN6T5YM7I2SLQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.740711713 (53h26m27.171791844s) [noise=true , address=104.36.82.36:4638 , piece_id=XUABMR23MOZG5QXDR2P3R5BSO5HHRZOP5QVSLKIZADSOZGZZIZNQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.776799260 (53h26m27.207879391s) [noise=true , address=45.91.116.14:4221 , piece_id=GAKJS3G242VOGC66CVIKCPTX7ITXYM2SQFVJPUAVJHJI36DIZ4BQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.823959072 (53h26m27.255039203s) [noise=true , address=104.36.82.44:4888 , piece_id=DPVGIMFES6RWUHO5OKRYJVX2HEDBBQJZHYYELVLMCBOWUKP6IKJA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.873921155 (53h26m27.305001286s) [noise=true , address=104.36.82.35:4583 , piece_id=IXCJX5XZWILRXQMF24MUABV3TV2TW23R3T73XO563FKSTCDCYIYQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886211631 (53h26m27.317291762s) [noise=true , address=104.36.82.45:4913 , piece_id=QA35NTBOIGIZWNNTBFZXECHK4SQUHBVAHWPCXNAH2YAAERZEJHMQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:32.886240236 (53h26m27.317320367s) [noise=true , address=104.36.82.47:4549 , piece_id=IOXOLG2NU2H5CZIEXKPLH4EIJVKK4GX66SZQPDSCNPPKSOGFTUYA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:33.361231516 (53h26m27.792311647s) [noise=true , address=104.36.82.9:4030 , piece_id=7NIT67HJX2R7FUFLSSW36R54PJFKCGAKM5YJL6QWTO46UAY7GI5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.095423782 (53h26m28.526503913s) [noise=true , address=104.36.82.20:4453 , piece_id=LHTZE3WPC3N5RCNUZKGMPKW3EYUR2FPVB2G6DFL6PL4ZECRYBAIA, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.191766598 (53h26m28.622846729s) [noise=true , address=104.36.82.61:4378 , piece_id=KHNTUED6WVXAB6A7NZOE3JIUMY6FPQOJECGFWQG6ATWVCKMW2NVQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957234535 (53h26m29.388314666s) [noise=true , address=104.36.82.39:4746 , piece_id=UY4KKODNGNKAFOULMRSDCYJDT5HW7C6KYUD4EKZY3SBLXZRALOTQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: <nil> Aug 10 22:52:34.957270263 (53h26m29.388350384s) [seg_pos={0 0}, upload=55724]: Segment reached optimal threshold of 14 pieces. Aug 10 22:52:34.957519320 (53h26m29.388599441s) [noise=true , address=104.36.82.60:4348 , piece_id=Z4WXOGSIHUUIZPFDCJYA7ACUQXPXZWCT5QH7QLXR65T6AP2HIBIQ, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ezboNAWPbaK9UDENYoBb3iHjNU2ahn7mikiteN7y3Aj6uw4Nx): protocol: context canceled; context canceled Aug 10 22:52:34.957638617 (53h26m29.388718748s) [noise=true , address=104.36.82.37:4663 , piece_id=3IGY7SFDVZYNVVGICK3C2XRBA5LRW5OQVZDFKODY5LKW57HWUR7A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SwpAaXTA3BqUvwubzmJkvwokdz6SSpT9t8HPtQcXXJxmcgfm2): protocol: read tcp 188.166.225.170:38236->104.36.82.37:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957696528 (53h26m29.388776659s) [noise=true , address=104.36.82.56:4238 , piece_id=7YLBQEL63SEYD2OILQ2PRKEP7U46PT4PSPAASSV5DJPAYTESOJ5A, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12U2euZtUdrPZGo2PieqDnKtSftgtfdTRk6V8VfYS4yUW5z3qM5): protocol: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55512->104.36.82.56:4238: use of closed network connection Aug 10 22:52:34.957729272 (53h26m29.388809393s) [noise=true , address=104.36.82.8:4995 , piece_id=G3K3LCGGAPEURR4G7HCBRM36ZBI4Q7MY2X5YT4CJLS2LRDRKXE7Q, seg_pos={0 0}, upload=55724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tvoTD1LoSj3q2rLiDckfT56dXNKKwdVu1ZAsBZExNFVTom9tH): protocol: rpc: dial tcp 104.36.82.8:4995: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 10 22:52:34.957748227 (53h26m29.388828348s) [seg_pos={0 0}, upload=55724]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 10 22:52:34.957846646 (53h26m29.388926767s) [upload=55724]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 10 22:52:34.957851144 (53h26m29.388931275s) [upload=55724]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:52:35.306840333 (53h26m29.737920484s) [upload=55724]: Done uploading segments... Aug 10 22:52:35.306857145 (53h26m29.737937276s) [upload=55724]: Upload finished. err: <nil> Duration: 7.234761845s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 1.966796293s FAIL (2025-08-10 22:56:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 10 22:33:23.090759975 (53h7m17.521840126s) [upload=55388]: Starting upload Aug 10 22:33:23.090831261 (53h7m17.521911422s) [upload=55388]: Done starting upload Aug 10 22:33:23.090903269 (53h7m17.52198342s) [upload=55388]: Uploading segments... Aug 10 22:33:23.090955248 (53h7m17.52203541s) [upload=55388]: Got next segment. Inline: true Aug 10 22:33:23.090961811 (53h7m17.522041962s) [upload=55388]: Waiting for error group managing segments... Aug 10 22:33:23.091016636 (53h7m17.522096807s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:33:23.091023408 (53h7m17.52210356s) [upload=55388]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 22:33:23.091028138 (53h7m17.522108289s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:33:23.358287015 (53h7m17.789367146s) [upload=55388]: Done uploading segments... Aug 10 22:33:23.358303086 (53h7m17.789383207s) [upload=55388]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.966822474s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 2.040794997s FAIL (2025-08-10 22:55:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 10 22:33:23.090759975 (53h7m17.521840126s) [upload=55388]: Starting upload Aug 10 22:33:23.090831261 (53h7m17.521911422s) [upload=55388]: Done starting upload Aug 10 22:33:23.090903269 (53h7m17.52198342s) [upload=55388]: Uploading segments... Aug 10 22:33:23.090955248 (53h7m17.52203541s) [upload=55388]: Got next segment. Inline: true Aug 10 22:33:23.090961811 (53h7m17.522041962s) [upload=55388]: Waiting for error group managing segments... Aug 10 22:33:23.091016636 (53h7m17.522096807s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 10 22:33:23.091023408 (53h7m17.52210356s) [upload=55388]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 10 22:33:23.091028138 (53h7m17.522108289s) [upload=55388]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 10 22:33:23.358287015 (53h7m17.789367146s) [upload=55388]: Done uploading segments... Aug 10 22:33:23.358303086 (53h7m17.789383207s) [upload=55388]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.040824885s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)