Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..........................X.X.........X..........X.....X....] duration: 6.290853351s FAIL (2025-08-12 17:17:41 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 12 17:13:28.825252727 (95h47m23.256332898s) [upload=99787]: Starting upload Aug 12 17:13:28.825321419 (95h47m23.25640159s) [upload=99787]: Done starting upload Aug 12 17:13:28.825449604 (95h47m23.256529775s) [upload=99787]: Uploading segments... Aug 12 17:13:28.825744037 (95h47m23.256824198s) [upload=99787]: Got next segment. Inline: false Aug 12 17:13:28.825821235 (95h47m23.256901416s) [upload=99787]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 17:13:28.825829931 (95h47m23.256910093s) [upload=99787]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 17:13:29.303728856 (95h47m23.734808987s) [seg_pos={0 0}, upload=99787]: Begin upload segment... Aug 12 17:13:29.303773140 (95h47m23.734853271s) [seg_pos={0 0}, upload=99787]: Done begin upload segment. Aug 12 17:13:29.303793930 (95h47m23.734874061s) [noise=true , address=104.36.82.44:4900 , piece_id=LCDUXPUFOZ43NKZ4LOR5ZCVQ3RPPCTIR2UU7XDNNFHHXDS77UKFQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.303866910 (95h47m23.734947041s) [noise=true , address=104.36.82.58:4266 , piece_id=XOTBH7GZGN7QZFS2U2AS72TKR3AU2KYQD474Y6GMFFUYT4LHPLHQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.304176021 (95h47m23.735256152s) [noise=true , address=104.36.82.38:4694 , piece_id=RZ6LNKXDHDRU73N6NIULQG34PVWCYWIVA4FJ2BEF3CKUCFXGAKMQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.304419978 (95h47m23.735500109s) [noise=true , address=104.36.82.9:4025 , piece_id=AQJMXPICHPUI5UPVL6WZWMGZ7UFBVDK5KS4IFV3XBOXX76DMJVRA, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.304639759 (95h47m23.73571988s) [noise=true , address=79.127.219.51:4195 , piece_id=4EU53SFRSCZ2NNZBFWI427YQHR7BEFT5JX6TW7E33P2TRIN4FQXQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.304780888 (95h47m23.735861019s) [noise=true , address=64.49.13.154:4936 , piece_id=FF3TYKME6R42QWQEW27PSKJ5ORUC4A7SMWVHPH6L2ODX3TISQH5A, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.304926106 (95h47m23.736006227s) [noise=true , address=104.36.82.62:4441 , piece_id=MC7SVGPDMN4VVKEOB5CEERHLNI53KA6XGSWXE6OQWCMEL2XTWVJA, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.305191725 (95h47m23.736271856s) [noise=true , address=104.36.82.56:4237 , piece_id=BV33QX3IWIUX45JH7NJPW6A2BYF2CWF3PYVPC4YJQ75KCZ3JEXVA, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.305257680 (95h47m23.736337821s) [noise=true , address=64.49.13.140:4506 , piece_id=SSZ5XSTLM4CKZTRLABB2CC5BK54E3BGYX5HWWLXRNVK5BIPRVEFQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.305590016 (95h47m23.736670147s) [noise=true , address=104.36.82.19:4402 , piece_id=IEL4AM22HO26IC5TSAFKXCMHY3B4MGPOTDD2HVQPYEFD3GCJCADQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.305886914 (95h47m23.736967035s) [noise=true , address=104.36.82.17:4345 , piece_id=VVRYNEI7FMSK2QHNR7N2EACZQZVBKG54ADWESJNGS43PVEG5EMLQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.305961698 (95h47m23.737041829s) [noise=true , address=104.36.82.45:4931 , piece_id=WKX4RE7H2PMKMEZWTKT6EF2BALALCT2ZPNFAUFMC7IFUJITEJ7AQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306106645 (95h47m23.737186776s) [noise=true , address=104.36.82.7:4915 , piece_id=35GNLEYYZL7J2RXDJ6ONZOX3SHEJHUCKVJDAGV64KWKURW5PCTNQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306212778 (95h47m23.737292909s) [noise=true , address=104.36.82.11:4142 , piece_id=7D5FHBGX6S6CTMTK6IEUEYNOT73LQXJYZV6HMUVVP6OOPNR46R6Q, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306306056 (95h47m23.737386187s) [noise=true , address=104.36.82.39:4736 , piece_id=77BJAQPGH6SJN3D3K5UPCA2DNG4T7SK5ECK6ZN2LSVK46EYRXAEQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306459700 (95h47m23.737539832s) [noise=true , address=104.36.82.17:4344 , piece_id=PN5TP33O3LL7W5QJKB24GTSPNXBBKXIGETVIHSBRG4O252NY3T3A, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306513974 (95h47m23.737594105s) [noise=true , address=104.36.82.60:4348 , piece_id=ARK7NJB5532ZZHNVMVYG3XCBOUT4JCO5CEBRC2365T66ERZSTJEA, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306714267 (95h47m23.737794398s) [noise=true , address=77.90.31.16:4764 , piece_id=PVK7KVJD4ZTS2EA2FOU5APWROVZNYCZHGN3M4BXRHU4QJXGDGWWA, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:30.465351213 (95h47m24.896431344s) [upload=99787]: Next returned nil segment Aug 12 17:13:30.465368435 (95h47m24.896448567s) [upload=99787]: Waiting for error group managing segments... Aug 12 17:13:30.784923783 (95h47m25.216003914s) [noise=true , address=64.49.13.140:4506 , piece_id=SSZ5XSTLM4CKZTRLABB2CC5BK54E3BGYX5HWWLXRNVK5BIPRVEFQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:30.790033850 (95h47m25.221113981s) [noise=true , address=64.49.13.154:4936 , piece_id=FF3TYKME6R42QWQEW27PSKJ5ORUC4A7SMWVHPH6L2ODX3TISQH5A, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.255533719 (95h47m25.68661386s) [noise=true , address=104.36.82.38:4694 , piece_id=RZ6LNKXDHDRU73N6NIULQG34PVWCYWIVA4FJ2BEF3CKUCFXGAKMQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.269997042 (95h47m25.701077183s) [noise=true , address=104.36.82.60:4348 , piece_id=ARK7NJB5532ZZHNVMVYG3XCBOUT4JCO5CEBRC2365T66ERZSTJEA, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.302029471 (95h47m25.733109622s) [noise=true , address=77.90.31.16:4764 , piece_id=PVK7KVJD4ZTS2EA2FOU5APWROVZNYCZHGN3M4BXRHU4QJXGDGWWA, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.318353594 (95h47m25.749433725s) [noise=true , address=104.36.82.19:4402 , piece_id=IEL4AM22HO26IC5TSAFKXCMHY3B4MGPOTDD2HVQPYEFD3GCJCADQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.346017534 (95h47m25.777097665s) [noise=true , address=104.36.82.9:4025 , piece_id=AQJMXPICHPUI5UPVL6WZWMGZ7UFBVDK5KS4IFV3XBOXX76DMJVRA, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.348171306 (95h47m25.779251437s) [noise=true , address=104.36.82.45:4931 , piece_id=WKX4RE7H2PMKMEZWTKT6EF2BALALCT2ZPNFAUFMC7IFUJITEJ7AQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.354598163 (95h47m25.785678294s) [noise=true , address=104.36.82.56:4237 , piece_id=BV33QX3IWIUX45JH7NJPW6A2BYF2CWF3PYVPC4YJQ75KCZ3JEXVA, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.360073037 (95h47m25.791153169s) [noise=true , address=104.36.82.17:4345 , piece_id=VVRYNEI7FMSK2QHNR7N2EACZQZVBKG54ADWESJNGS43PVEG5EMLQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.371327111 (95h47m25.802407242s) [noise=true , address=104.36.82.39:4736 , piece_id=77BJAQPGH6SJN3D3K5UPCA2DNG4T7SK5ECK6ZN2LSVK46EYRXAEQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.372622479 (95h47m25.8037026s) [noise=true , address=104.36.82.44:4900 , piece_id=LCDUXPUFOZ43NKZ4LOR5ZCVQ3RPPCTIR2UU7XDNNFHHXDS77UKFQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.379217397 (95h47m25.810297528s) [noise=true , address=104.36.82.17:4344 , piece_id=PN5TP33O3LL7W5QJKB24GTSPNXBBKXIGETVIHSBRG4O252NY3T3A, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.393498702 (95h47m25.824578823s) [noise=true , address=104.36.82.58:4266 , piece_id=XOTBH7GZGN7QZFS2U2AS72TKR3AU2KYQD474Y6GMFFUYT4LHPLHQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.393527026 (95h47m25.824607157s) [seg_pos={0 0}, upload=99787]: Segment reached optimal threshold of 14 pieces. Aug 12 17:13:31.393647137 (95h47m25.824727268s) [noise=true , address=104.36.82.11:4142 , piece_id=7D5FHBGX6S6CTMTK6IEUEYNOT73LQXJYZV6HMUVVP6OOPNR46R6Q, seg_pos={0 0}, upload=99787]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JR1vFqWgTH1qwvxH4DEk4eKT7TfxWC2Re6QEWmqxWXctytfa7): protocol: expected piece hash; context canceled; EOF Aug 12 17:13:31.393705218 (95h47m25.824785339s) [noise=true , address=79.127.219.51:4195 , piece_id=4EU53SFRSCZ2NNZBFWI427YQHR7BEFT5JX6TW7E33P2TRIN4FQXQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mFXuncCmi6nsnuVqGK9QxfGCzGXgX3kcPPy9icg1fPemHpsx): protocol: expected piece hash; context canceled; EOF Aug 12 17:13:31.393756705 (95h47m25.824836836s) [noise=true , address=104.36.82.62:4441 , piece_id=MC7SVGPDMN4VVKEOB5CEERHLNI53KA6XGSWXE6OQWCMEL2XTWVJA, seg_pos={0 0}, upload=99787]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VBayWisMK1n3cwypeiLzGuffonBTzYga7thAZ8pSqrusc7kow): protocol: expected piece hash; context canceled; EOF Aug 12 17:13:31.393778618 (95h47m25.824858739s) [noise=true , address=104.36.82.7:4915 , piece_id=35GNLEYYZL7J2RXDJ6ONZOX3SHEJHUCKVJDAGV64KWKURW5PCTNQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt): protocol: expected piece hash; context canceled; EOF Aug 12 17:13:31.393833052 (95h47m25.824913183s) [seg_pos={0 0}, upload=99787]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 12 17:13:31.393916311 (95h47m25.824996442s) [upload=99787]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 17:13:31.393920689 (95h47m25.82500082s) [upload=99787]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 17:13:31.701564897 (95h47m26.132645038s) [upload=99787]: Done uploading segments... Aug 12 17:13:31.701588933 (95h47m26.132669114s) [upload=99787]: Upload finished. err: <nil> Duration: 6.290879532s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.930699191s FAIL (2025-08-12 17:17:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 12 16:51:00.342944596 (95h24m54.774024727s) [upload=99416]: Starting upload Aug 12 16:51:00.343003058 (95h24m54.774083189s) [upload=99416]: Done starting upload Aug 12 16:51:00.343093682 (95h24m54.774173803s) [upload=99416]: Uploading segments... Aug 12 16:51:00.343214823 (95h24m54.774294954s) [upload=99416]: Got next segment. Inline: false Aug 12 16:51:00.343269027 (95h24m54.774349158s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:51:00.343273756 (95h24m54.774353887s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 16:51:00.585349151 (95h24m55.016429282s) [seg_pos={0 0}, upload=99416]: Begin upload segment... Aug 12 16:51:00.585396822 (95h24m55.016476973s) [seg_pos={0 0}, upload=99416]: Done begin upload segment. Aug 12 16:51:00.585410067 (95h24m55.016490208s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585507734 (95h24m55.016587895s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585886159 (95h24m55.0169663s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586089607 (95h24m55.017169738s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586541652 (95h24m55.017621803s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587004688 (95h24m55.018084819s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587619234 (95h24m55.018699385s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587983842 (95h24m55.019063983s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588303153 (95h24m55.019383284s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588688090 (95h24m55.019768221s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.589902393 (95h24m55.020982534s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.590674450 (95h24m55.021754601s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.591296961 (95h24m55.022377092s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.592189969 (95h24m55.02327011s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.593156619 (95h24m55.02423678s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.594056290 (95h24m55.025136421s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:02.362133336 (95h24m56.793213467s) [upload=99416]: Next returned nil segment Aug 12 16:51:02.362159547 (95h24m56.793239678s) [upload=99416]: Waiting for error group managing segments... Aug 12 16:51:02.650293798 (95h24m57.081373929s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.663772758 (95h24m57.094852919s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.696344749 (95h24m57.12742488s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.709104884 (95h24m57.140185025s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.722233484 (95h24m57.153313625s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.727048907 (95h24m57.158129038s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.741870376 (95h24m57.172950507s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.772732365 (95h24m57.203812507s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.805755481 (95h24m57.236835612s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.878280266 (95h24m57.309360397s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.913734542 (95h24m57.344814673s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.972369120 (95h24m57.403449251s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983815812 (95h24m57.414895943s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983842733 (95h24m57.414922864s) [seg_pos={0 0}, upload=99416]: Segment reached optimal threshold of 13 pieces. Aug 12 16:51:02.983942084 (95h24m57.415022215s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.983999583 (95h24m57.415079715s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984087211 (95h24m57.415167342s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984124363 (95h24m57.415204494s) [seg_pos={0 0}, upload=99416]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 12 16:51:02.984229683 (95h24m57.415309814s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 16:51:02.984234323 (95h24m57.415314454s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:51:03.289516404 (95h24m57.720596575s) [upload=99416]: Done uploading segments... Aug 12 16:51:03.289540049 (95h24m57.7206202s) [upload=99416]: Upload finished. err: <nil> Duration: 5.930726612s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........X...........................................X.....] duration: 1.987721612s FAIL (2025-08-12 17:17:28 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 12 17:14:10.610708770 (95h48m5.041788891s) [upload=99796]: Starting upload Aug 12 17:14:10.610760419 (95h48m5.04184055s) [upload=99796]: Done starting upload Aug 12 17:14:10.610837096 (95h48m5.041917227s) [upload=99796]: Uploading segments... Aug 12 17:14:10.610918822 (95h48m5.041998954s) [upload=99796]: Got next segment. Inline: true Aug 12 17:14:10.610924463 (95h48m5.042004594s) [upload=99796]: Waiting for error group managing segments... Aug 12 17:14:10.610977775 (95h48m5.042057906s) [upload=99796]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 17:14:10.610983366 (95h48m5.042063497s) [upload=99796]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 17:14:10.610985930 (95h48m5.042066051s) [upload=99796]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 17:14:10.896688638 (95h48m5.327768759s) [upload=99796]: Done uploading segments... Aug 12 17:14:10.896709046 (95h48m5.327789177s) [upload=99796]: Upload finished. err: <nil> Duration:* 1.98775143s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [X..........X...........................................X....] duration: 1.990331156s FAIL (2025-08-12 17:16:59 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 12 17:14:10.610708770 (95h48m5.041788891s) [upload=99796]: Starting upload Aug 12 17:14:10.610760419 (95h48m5.04184055s) [upload=99796]: Done starting upload Aug 12 17:14:10.610837096 (95h48m5.041917227s) [upload=99796]: Uploading segments... Aug 12 17:14:10.610918822 (95h48m5.041998954s) [upload=99796]: Got next segment. Inline: true Aug 12 17:14:10.610924463 (95h48m5.042004594s) [upload=99796]: Waiting for error group managing segments... Aug 12 17:14:10.610977775 (95h48m5.042057906s) [upload=99796]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 17:14:10.610983366 (95h48m5.042063497s) [upload=99796]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 17:14:10.610985930 (95h48m5.042066051s) [upload=99796]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 17:14:10.896688638 (95h48m5.327768759s) [upload=99796]: Done uploading segments... Aug 12 17:14:10.896709046 (95h48m5.327789177s) [upload=99796]: Upload finished. err: <nil> Duration:* 1.99034933s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.830970907s FAIL (2025-08-12 17:16:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 12 16:51:00.342944596 (95h24m54.774024727s) [upload=99416]: Starting upload Aug 12 16:51:00.343003058 (95h24m54.774083189s) [upload=99416]: Done starting upload Aug 12 16:51:00.343093682 (95h24m54.774173803s) [upload=99416]: Uploading segments... Aug 12 16:51:00.343214823 (95h24m54.774294954s) [upload=99416]: Got next segment. Inline: false Aug 12 16:51:00.343269027 (95h24m54.774349158s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:51:00.343273756 (95h24m54.774353887s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 16:51:00.585349151 (95h24m55.016429282s) [seg_pos={0 0}, upload=99416]: Begin upload segment... Aug 12 16:51:00.585396822 (95h24m55.016476973s) [seg_pos={0 0}, upload=99416]: Done begin upload segment. Aug 12 16:51:00.585410067 (95h24m55.016490208s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585507734 (95h24m55.016587895s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585886159 (95h24m55.0169663s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586089607 (95h24m55.017169738s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586541652 (95h24m55.017621803s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587004688 (95h24m55.018084819s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587619234 (95h24m55.018699385s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587983842 (95h24m55.019063983s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588303153 (95h24m55.019383284s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588688090 (95h24m55.019768221s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.589902393 (95h24m55.020982534s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.590674450 (95h24m55.021754601s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.591296961 (95h24m55.022377092s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.592189969 (95h24m55.02327011s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.593156619 (95h24m55.02423678s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.594056290 (95h24m55.025136421s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:02.362133336 (95h24m56.793213467s) [upload=99416]: Next returned nil segment Aug 12 16:51:02.362159547 (95h24m56.793239678s) [upload=99416]: Waiting for error group managing segments... Aug 12 16:51:02.650293798 (95h24m57.081373929s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.663772758 (95h24m57.094852919s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.696344749 (95h24m57.12742488s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.709104884 (95h24m57.140185025s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.722233484 (95h24m57.153313625s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.727048907 (95h24m57.158129038s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.741870376 (95h24m57.172950507s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.772732365 (95h24m57.203812507s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.805755481 (95h24m57.236835612s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.878280266 (95h24m57.309360397s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.913734542 (95h24m57.344814673s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.972369120 (95h24m57.403449251s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983815812 (95h24m57.414895943s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983842733 (95h24m57.414922864s) [seg_pos={0 0}, upload=99416]: Segment reached optimal threshold of 13 pieces. Aug 12 16:51:02.983942084 (95h24m57.415022215s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.983999583 (95h24m57.415079715s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984087211 (95h24m57.415167342s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984124363 (95h24m57.415204494s) [seg_pos={0 0}, upload=99416]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 12 16:51:02.984229683 (95h24m57.415309814s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 16:51:02.984234323 (95h24m57.415314454s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:51:03.289516404 (95h24m57.720596575s) [upload=99416]: Done uploading segments... Aug 12 16:51:03.289540049 (95h24m57.7206202s) [upload=99416]: Upload finished. err: <nil> Duration: 5.831016024s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.X..........X...........................................X...] duration: 1.902535785s FAIL (2025-08-12 17:16:37 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 12 17:14:10.610708770 (95h48m5.041788891s) [upload=99796]: Starting upload Aug 12 17:14:10.610760419 (95h48m5.04184055s) [upload=99796]: Done starting upload Aug 12 17:14:10.610837096 (95h48m5.041917227s) [upload=99796]: Uploading segments... Aug 12 17:14:10.610918822 (95h48m5.041998954s) [upload=99796]: Got next segment. Inline: true Aug 12 17:14:10.610924463 (95h48m5.042004594s) [upload=99796]: Waiting for error group managing segments... Aug 12 17:14:10.610977775 (95h48m5.042057906s) [upload=99796]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 17:14:10.610983366 (95h48m5.042063497s) [upload=99796]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 17:14:10.610985930 (95h48m5.042066051s) [upload=99796]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 17:14:10.896688638 (95h48m5.327768759s) [upload=99796]: Done uploading segments... Aug 12 17:14:10.896709046 (95h48m5.327789177s) [upload=99796]: Upload finished. err: <nil> Duration:* 1.902556564s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...........................X.X.........X..........X.....X...] duration: 5.942449068s FAIL (2025-08-12 17:16:27 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 12 17:13:28.825252727 (95h47m23.256332898s) [upload=99787]: Starting upload Aug 12 17:13:28.825321419 (95h47m23.25640159s) [upload=99787]: Done starting upload Aug 12 17:13:28.825449604 (95h47m23.256529775s) [upload=99787]: Uploading segments... Aug 12 17:13:28.825744037 (95h47m23.256824198s) [upload=99787]: Got next segment. Inline: false Aug 12 17:13:28.825821235 (95h47m23.256901416s) [upload=99787]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 17:13:28.825829931 (95h47m23.256910093s) [upload=99787]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 17:13:29.303728856 (95h47m23.734808987s) [seg_pos={0 0}, upload=99787]: Begin upload segment... Aug 12 17:13:29.303773140 (95h47m23.734853271s) [seg_pos={0 0}, upload=99787]: Done begin upload segment. Aug 12 17:13:29.303793930 (95h47m23.734874061s) [noise=true , address=104.36.82.44:4900 , piece_id=LCDUXPUFOZ43NKZ4LOR5ZCVQ3RPPCTIR2UU7XDNNFHHXDS77UKFQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.303866910 (95h47m23.734947041s) [noise=true , address=104.36.82.58:4266 , piece_id=XOTBH7GZGN7QZFS2U2AS72TKR3AU2KYQD474Y6GMFFUYT4LHPLHQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.304176021 (95h47m23.735256152s) [noise=true , address=104.36.82.38:4694 , piece_id=RZ6LNKXDHDRU73N6NIULQG34PVWCYWIVA4FJ2BEF3CKUCFXGAKMQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.304419978 (95h47m23.735500109s) [noise=true , address=104.36.82.9:4025 , piece_id=AQJMXPICHPUI5UPVL6WZWMGZ7UFBVDK5KS4IFV3XBOXX76DMJVRA, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.304639759 (95h47m23.73571988s) [noise=true , address=79.127.219.51:4195 , piece_id=4EU53SFRSCZ2NNZBFWI427YQHR7BEFT5JX6TW7E33P2TRIN4FQXQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.304780888 (95h47m23.735861019s) [noise=true , address=64.49.13.154:4936 , piece_id=FF3TYKME6R42QWQEW27PSKJ5ORUC4A7SMWVHPH6L2ODX3TISQH5A, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.304926106 (95h47m23.736006227s) [noise=true , address=104.36.82.62:4441 , piece_id=MC7SVGPDMN4VVKEOB5CEERHLNI53KA6XGSWXE6OQWCMEL2XTWVJA, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.305191725 (95h47m23.736271856s) [noise=true , address=104.36.82.56:4237 , piece_id=BV33QX3IWIUX45JH7NJPW6A2BYF2CWF3PYVPC4YJQ75KCZ3JEXVA, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.305257680 (95h47m23.736337821s) [noise=true , address=64.49.13.140:4506 , piece_id=SSZ5XSTLM4CKZTRLABB2CC5BK54E3BGYX5HWWLXRNVK5BIPRVEFQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.305590016 (95h47m23.736670147s) [noise=true , address=104.36.82.19:4402 , piece_id=IEL4AM22HO26IC5TSAFKXCMHY3B4MGPOTDD2HVQPYEFD3GCJCADQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.305886914 (95h47m23.736967035s) [noise=true , address=104.36.82.17:4345 , piece_id=VVRYNEI7FMSK2QHNR7N2EACZQZVBKG54ADWESJNGS43PVEG5EMLQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.305961698 (95h47m23.737041829s) [noise=true , address=104.36.82.45:4931 , piece_id=WKX4RE7H2PMKMEZWTKT6EF2BALALCT2ZPNFAUFMC7IFUJITEJ7AQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306106645 (95h47m23.737186776s) [noise=true , address=104.36.82.7:4915 , piece_id=35GNLEYYZL7J2RXDJ6ONZOX3SHEJHUCKVJDAGV64KWKURW5PCTNQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306212778 (95h47m23.737292909s) [noise=true , address=104.36.82.11:4142 , piece_id=7D5FHBGX6S6CTMTK6IEUEYNOT73LQXJYZV6HMUVVP6OOPNR46R6Q, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306306056 (95h47m23.737386187s) [noise=true , address=104.36.82.39:4736 , piece_id=77BJAQPGH6SJN3D3K5UPCA2DNG4T7SK5ECK6ZN2LSVK46EYRXAEQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306459700 (95h47m23.737539832s) [noise=true , address=104.36.82.17:4344 , piece_id=PN5TP33O3LL7W5QJKB24GTSPNXBBKXIGETVIHSBRG4O252NY3T3A, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306513974 (95h47m23.737594105s) [noise=true , address=104.36.82.60:4348 , piece_id=ARK7NJB5532ZZHNVMVYG3XCBOUT4JCO5CEBRC2365T66ERZSTJEA, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306714267 (95h47m23.737794398s) [noise=true , address=77.90.31.16:4764 , piece_id=PVK7KVJD4ZTS2EA2FOU5APWROVZNYCZHGN3M4BXRHU4QJXGDGWWA, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:30.465351213 (95h47m24.896431344s) [upload=99787]: Next returned nil segment Aug 12 17:13:30.465368435 (95h47m24.896448567s) [upload=99787]: Waiting for error group managing segments... Aug 12 17:13:30.784923783 (95h47m25.216003914s) [noise=true , address=64.49.13.140:4506 , piece_id=SSZ5XSTLM4CKZTRLABB2CC5BK54E3BGYX5HWWLXRNVK5BIPRVEFQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:30.790033850 (95h47m25.221113981s) [noise=true , address=64.49.13.154:4936 , piece_id=FF3TYKME6R42QWQEW27PSKJ5ORUC4A7SMWVHPH6L2ODX3TISQH5A, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.255533719 (95h47m25.68661386s) [noise=true , address=104.36.82.38:4694 , piece_id=RZ6LNKXDHDRU73N6NIULQG34PVWCYWIVA4FJ2BEF3CKUCFXGAKMQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.269997042 (95h47m25.701077183s) [noise=true , address=104.36.82.60:4348 , piece_id=ARK7NJB5532ZZHNVMVYG3XCBOUT4JCO5CEBRC2365T66ERZSTJEA, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.302029471 (95h47m25.733109622s) [noise=true , address=77.90.31.16:4764 , piece_id=PVK7KVJD4ZTS2EA2FOU5APWROVZNYCZHGN3M4BXRHU4QJXGDGWWA, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.318353594 (95h47m25.749433725s) [noise=true , address=104.36.82.19:4402 , piece_id=IEL4AM22HO26IC5TSAFKXCMHY3B4MGPOTDD2HVQPYEFD3GCJCADQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.346017534 (95h47m25.777097665s) [noise=true , address=104.36.82.9:4025 , piece_id=AQJMXPICHPUI5UPVL6WZWMGZ7UFBVDK5KS4IFV3XBOXX76DMJVRA, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.348171306 (95h47m25.779251437s) [noise=true , address=104.36.82.45:4931 , piece_id=WKX4RE7H2PMKMEZWTKT6EF2BALALCT2ZPNFAUFMC7IFUJITEJ7AQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.354598163 (95h47m25.785678294s) [noise=true , address=104.36.82.56:4237 , piece_id=BV33QX3IWIUX45JH7NJPW6A2BYF2CWF3PYVPC4YJQ75KCZ3JEXVA, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.360073037 (95h47m25.791153169s) [noise=true , address=104.36.82.17:4345 , piece_id=VVRYNEI7FMSK2QHNR7N2EACZQZVBKG54ADWESJNGS43PVEG5EMLQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.371327111 (95h47m25.802407242s) [noise=true , address=104.36.82.39:4736 , piece_id=77BJAQPGH6SJN3D3K5UPCA2DNG4T7SK5ECK6ZN2LSVK46EYRXAEQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.372622479 (95h47m25.8037026s) [noise=true , address=104.36.82.44:4900 , piece_id=LCDUXPUFOZ43NKZ4LOR5ZCVQ3RPPCTIR2UU7XDNNFHHXDS77UKFQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.379217397 (95h47m25.810297528s) [noise=true , address=104.36.82.17:4344 , piece_id=PN5TP33O3LL7W5QJKB24GTSPNXBBKXIGETVIHSBRG4O252NY3T3A, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.393498702 (95h47m25.824578823s) [noise=true , address=104.36.82.58:4266 , piece_id=XOTBH7GZGN7QZFS2U2AS72TKR3AU2KYQD474Y6GMFFUYT4LHPLHQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.393527026 (95h47m25.824607157s) [seg_pos={0 0}, upload=99787]: Segment reached optimal threshold of 14 pieces. Aug 12 17:13:31.393647137 (95h47m25.824727268s) [noise=true , address=104.36.82.11:4142 , piece_id=7D5FHBGX6S6CTMTK6IEUEYNOT73LQXJYZV6HMUVVP6OOPNR46R6Q, seg_pos={0 0}, upload=99787]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JR1vFqWgTH1qwvxH4DEk4eKT7TfxWC2Re6QEWmqxWXctytfa7): protocol: expected piece hash; context canceled; EOF Aug 12 17:13:31.393705218 (95h47m25.824785339s) [noise=true , address=79.127.219.51:4195 , piece_id=4EU53SFRSCZ2NNZBFWI427YQHR7BEFT5JX6TW7E33P2TRIN4FQXQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mFXuncCmi6nsnuVqGK9QxfGCzGXgX3kcPPy9icg1fPemHpsx): protocol: expected piece hash; context canceled; EOF Aug 12 17:13:31.393756705 (95h47m25.824836836s) [noise=true , address=104.36.82.62:4441 , piece_id=MC7SVGPDMN4VVKEOB5CEERHLNI53KA6XGSWXE6OQWCMEL2XTWVJA, seg_pos={0 0}, upload=99787]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VBayWisMK1n3cwypeiLzGuffonBTzYga7thAZ8pSqrusc7kow): protocol: expected piece hash; context canceled; EOF Aug 12 17:13:31.393778618 (95h47m25.824858739s) [noise=true , address=104.36.82.7:4915 , piece_id=35GNLEYYZL7J2RXDJ6ONZOX3SHEJHUCKVJDAGV64KWKURW5PCTNQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt): protocol: expected piece hash; context canceled; EOF Aug 12 17:13:31.393833052 (95h47m25.824913183s) [seg_pos={0 0}, upload=99787]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 12 17:13:31.393916311 (95h47m25.824996442s) [upload=99787]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 17:13:31.393920689 (95h47m25.82500082s) [upload=99787]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 17:13:31.701564897 (95h47m26.132645038s) [upload=99787]: Done uploading segments... Aug 12 17:13:31.701588933 (95h47m26.132669114s) [upload=99787]: Upload finished. err: <nil> Duration: 5.942478593s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.874955666s FAIL (2025-08-12 17:15:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 12 16:51:00.342944596 (95h24m54.774024727s) [upload=99416]: Starting upload Aug 12 16:51:00.343003058 (95h24m54.774083189s) [upload=99416]: Done starting upload Aug 12 16:51:00.343093682 (95h24m54.774173803s) [upload=99416]: Uploading segments... Aug 12 16:51:00.343214823 (95h24m54.774294954s) [upload=99416]: Got next segment. Inline: false Aug 12 16:51:00.343269027 (95h24m54.774349158s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:51:00.343273756 (95h24m54.774353887s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 16:51:00.585349151 (95h24m55.016429282s) [seg_pos={0 0}, upload=99416]: Begin upload segment... Aug 12 16:51:00.585396822 (95h24m55.016476973s) [seg_pos={0 0}, upload=99416]: Done begin upload segment. Aug 12 16:51:00.585410067 (95h24m55.016490208s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585507734 (95h24m55.016587895s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585886159 (95h24m55.0169663s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586089607 (95h24m55.017169738s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586541652 (95h24m55.017621803s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587004688 (95h24m55.018084819s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587619234 (95h24m55.018699385s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587983842 (95h24m55.019063983s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588303153 (95h24m55.019383284s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588688090 (95h24m55.019768221s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.589902393 (95h24m55.020982534s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.590674450 (95h24m55.021754601s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.591296961 (95h24m55.022377092s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.592189969 (95h24m55.02327011s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.593156619 (95h24m55.02423678s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.594056290 (95h24m55.025136421s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:02.362133336 (95h24m56.793213467s) [upload=99416]: Next returned nil segment Aug 12 16:51:02.362159547 (95h24m56.793239678s) [upload=99416]: Waiting for error group managing segments... Aug 12 16:51:02.650293798 (95h24m57.081373929s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.663772758 (95h24m57.094852919s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.696344749 (95h24m57.12742488s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.709104884 (95h24m57.140185025s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.722233484 (95h24m57.153313625s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.727048907 (95h24m57.158129038s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.741870376 (95h24m57.172950507s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.772732365 (95h24m57.203812507s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.805755481 (95h24m57.236835612s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.878280266 (95h24m57.309360397s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.913734542 (95h24m57.344814673s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.972369120 (95h24m57.403449251s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983815812 (95h24m57.414895943s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983842733 (95h24m57.414922864s) [seg_pos={0 0}, upload=99416]: Segment reached optimal threshold of 13 pieces. Aug 12 16:51:02.983942084 (95h24m57.415022215s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.983999583 (95h24m57.415079715s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984087211 (95h24m57.415167342s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984124363 (95h24m57.415204494s) [seg_pos={0 0}, upload=99416]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 12 16:51:02.984229683 (95h24m57.415309814s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 16:51:02.984234323 (95h24m57.415314454s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:51:03.289516404 (95h24m57.720596575s) [upload=99416]: Done uploading segments... Aug 12 16:51:03.289540049 (95h24m57.7206202s) [upload=99416]: Upload finished. err: <nil> Duration: 5.874982479s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..X..........X...........................................X..] duration: 1.930498017s FAIL (2025-08-12 17:15:50 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 12 17:14:10.610708770 (95h48m5.041788891s) [upload=99796]: Starting upload Aug 12 17:14:10.610760419 (95h48m5.04184055s) [upload=99796]: Done starting upload Aug 12 17:14:10.610837096 (95h48m5.041917227s) [upload=99796]: Uploading segments... Aug 12 17:14:10.610918822 (95h48m5.041998954s) [upload=99796]: Got next segment. Inline: true Aug 12 17:14:10.610924463 (95h48m5.042004594s) [upload=99796]: Waiting for error group managing segments... Aug 12 17:14:10.610977775 (95h48m5.042057906s) [upload=99796]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 17:14:10.610983366 (95h48m5.042063497s) [upload=99796]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 17:14:10.610985930 (95h48m5.042066051s) [upload=99796]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 17:14:10.896688638 (95h48m5.327768759s) [upload=99796]: Done uploading segments... Aug 12 17:14:10.896709046 (95h48m5.327789177s) [upload=99796]: Upload finished. err: <nil> Duration:* 1.930525459s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [............................X.X.........X..........X.....X..] duration: 5.956874879s FAIL (2025-08-12 17:15:48 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 12 17:13:28.825252727 (95h47m23.256332898s) [upload=99787]: Starting upload Aug 12 17:13:28.825321419 (95h47m23.25640159s) [upload=99787]: Done starting upload Aug 12 17:13:28.825449604 (95h47m23.256529775s) [upload=99787]: Uploading segments... Aug 12 17:13:28.825744037 (95h47m23.256824198s) [upload=99787]: Got next segment. Inline: false Aug 12 17:13:28.825821235 (95h47m23.256901416s) [upload=99787]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 17:13:28.825829931 (95h47m23.256910093s) [upload=99787]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 17:13:29.303728856 (95h47m23.734808987s) [seg_pos={0 0}, upload=99787]: Begin upload segment... Aug 12 17:13:29.303773140 (95h47m23.734853271s) [seg_pos={0 0}, upload=99787]: Done begin upload segment. Aug 12 17:13:29.303793930 (95h47m23.734874061s) [noise=true , address=104.36.82.44:4900 , piece_id=LCDUXPUFOZ43NKZ4LOR5ZCVQ3RPPCTIR2UU7XDNNFHHXDS77UKFQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.303866910 (95h47m23.734947041s) [noise=true , address=104.36.82.58:4266 , piece_id=XOTBH7GZGN7QZFS2U2AS72TKR3AU2KYQD474Y6GMFFUYT4LHPLHQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.304176021 (95h47m23.735256152s) [noise=true , address=104.36.82.38:4694 , piece_id=RZ6LNKXDHDRU73N6NIULQG34PVWCYWIVA4FJ2BEF3CKUCFXGAKMQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.304419978 (95h47m23.735500109s) [noise=true , address=104.36.82.9:4025 , piece_id=AQJMXPICHPUI5UPVL6WZWMGZ7UFBVDK5KS4IFV3XBOXX76DMJVRA, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.304639759 (95h47m23.73571988s) [noise=true , address=79.127.219.51:4195 , piece_id=4EU53SFRSCZ2NNZBFWI427YQHR7BEFT5JX6TW7E33P2TRIN4FQXQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.304780888 (95h47m23.735861019s) [noise=true , address=64.49.13.154:4936 , piece_id=FF3TYKME6R42QWQEW27PSKJ5ORUC4A7SMWVHPH6L2ODX3TISQH5A, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.304926106 (95h47m23.736006227s) [noise=true , address=104.36.82.62:4441 , piece_id=MC7SVGPDMN4VVKEOB5CEERHLNI53KA6XGSWXE6OQWCMEL2XTWVJA, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.305191725 (95h47m23.736271856s) [noise=true , address=104.36.82.56:4237 , piece_id=BV33QX3IWIUX45JH7NJPW6A2BYF2CWF3PYVPC4YJQ75KCZ3JEXVA, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.305257680 (95h47m23.736337821s) [noise=true , address=64.49.13.140:4506 , piece_id=SSZ5XSTLM4CKZTRLABB2CC5BK54E3BGYX5HWWLXRNVK5BIPRVEFQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.305590016 (95h47m23.736670147s) [noise=true , address=104.36.82.19:4402 , piece_id=IEL4AM22HO26IC5TSAFKXCMHY3B4MGPOTDD2HVQPYEFD3GCJCADQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.305886914 (95h47m23.736967035s) [noise=true , address=104.36.82.17:4345 , piece_id=VVRYNEI7FMSK2QHNR7N2EACZQZVBKG54ADWESJNGS43PVEG5EMLQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.305961698 (95h47m23.737041829s) [noise=true , address=104.36.82.45:4931 , piece_id=WKX4RE7H2PMKMEZWTKT6EF2BALALCT2ZPNFAUFMC7IFUJITEJ7AQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306106645 (95h47m23.737186776s) [noise=true , address=104.36.82.7:4915 , piece_id=35GNLEYYZL7J2RXDJ6ONZOX3SHEJHUCKVJDAGV64KWKURW5PCTNQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306212778 (95h47m23.737292909s) [noise=true , address=104.36.82.11:4142 , piece_id=7D5FHBGX6S6CTMTK6IEUEYNOT73LQXJYZV6HMUVVP6OOPNR46R6Q, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306306056 (95h47m23.737386187s) [noise=true , address=104.36.82.39:4736 , piece_id=77BJAQPGH6SJN3D3K5UPCA2DNG4T7SK5ECK6ZN2LSVK46EYRXAEQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306459700 (95h47m23.737539832s) [noise=true , address=104.36.82.17:4344 , piece_id=PN5TP33O3LL7W5QJKB24GTSPNXBBKXIGETVIHSBRG4O252NY3T3A, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306513974 (95h47m23.737594105s) [noise=true , address=104.36.82.60:4348 , piece_id=ARK7NJB5532ZZHNVMVYG3XCBOUT4JCO5CEBRC2365T66ERZSTJEA, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306714267 (95h47m23.737794398s) [noise=true , address=77.90.31.16:4764 , piece_id=PVK7KVJD4ZTS2EA2FOU5APWROVZNYCZHGN3M4BXRHU4QJXGDGWWA, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:30.465351213 (95h47m24.896431344s) [upload=99787]: Next returned nil segment Aug 12 17:13:30.465368435 (95h47m24.896448567s) [upload=99787]: Waiting for error group managing segments... Aug 12 17:13:30.784923783 (95h47m25.216003914s) [noise=true , address=64.49.13.140:4506 , piece_id=SSZ5XSTLM4CKZTRLABB2CC5BK54E3BGYX5HWWLXRNVK5BIPRVEFQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:30.790033850 (95h47m25.221113981s) [noise=true , address=64.49.13.154:4936 , piece_id=FF3TYKME6R42QWQEW27PSKJ5ORUC4A7SMWVHPH6L2ODX3TISQH5A, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.255533719 (95h47m25.68661386s) [noise=true , address=104.36.82.38:4694 , piece_id=RZ6LNKXDHDRU73N6NIULQG34PVWCYWIVA4FJ2BEF3CKUCFXGAKMQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.269997042 (95h47m25.701077183s) [noise=true , address=104.36.82.60:4348 , piece_id=ARK7NJB5532ZZHNVMVYG3XCBOUT4JCO5CEBRC2365T66ERZSTJEA, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.302029471 (95h47m25.733109622s) [noise=true , address=77.90.31.16:4764 , piece_id=PVK7KVJD4ZTS2EA2FOU5APWROVZNYCZHGN3M4BXRHU4QJXGDGWWA, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.318353594 (95h47m25.749433725s) [noise=true , address=104.36.82.19:4402 , piece_id=IEL4AM22HO26IC5TSAFKXCMHY3B4MGPOTDD2HVQPYEFD3GCJCADQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.346017534 (95h47m25.777097665s) [noise=true , address=104.36.82.9:4025 , piece_id=AQJMXPICHPUI5UPVL6WZWMGZ7UFBVDK5KS4IFV3XBOXX76DMJVRA, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.348171306 (95h47m25.779251437s) [noise=true , address=104.36.82.45:4931 , piece_id=WKX4RE7H2PMKMEZWTKT6EF2BALALCT2ZPNFAUFMC7IFUJITEJ7AQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.354598163 (95h47m25.785678294s) [noise=true , address=104.36.82.56:4237 , piece_id=BV33QX3IWIUX45JH7NJPW6A2BYF2CWF3PYVPC4YJQ75KCZ3JEXVA, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.360073037 (95h47m25.791153169s) [noise=true , address=104.36.82.17:4345 , piece_id=VVRYNEI7FMSK2QHNR7N2EACZQZVBKG54ADWESJNGS43PVEG5EMLQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.371327111 (95h47m25.802407242s) [noise=true , address=104.36.82.39:4736 , piece_id=77BJAQPGH6SJN3D3K5UPCA2DNG4T7SK5ECK6ZN2LSVK46EYRXAEQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.372622479 (95h47m25.8037026s) [noise=true , address=104.36.82.44:4900 , piece_id=LCDUXPUFOZ43NKZ4LOR5ZCVQ3RPPCTIR2UU7XDNNFHHXDS77UKFQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.379217397 (95h47m25.810297528s) [noise=true , address=104.36.82.17:4344 , piece_id=PN5TP33O3LL7W5QJKB24GTSPNXBBKXIGETVIHSBRG4O252NY3T3A, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.393498702 (95h47m25.824578823s) [noise=true , address=104.36.82.58:4266 , piece_id=XOTBH7GZGN7QZFS2U2AS72TKR3AU2KYQD474Y6GMFFUYT4LHPLHQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.393527026 (95h47m25.824607157s) [seg_pos={0 0}, upload=99787]: Segment reached optimal threshold of 14 pieces. Aug 12 17:13:31.393647137 (95h47m25.824727268s) [noise=true , address=104.36.82.11:4142 , piece_id=7D5FHBGX6S6CTMTK6IEUEYNOT73LQXJYZV6HMUVVP6OOPNR46R6Q, seg_pos={0 0}, upload=99787]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JR1vFqWgTH1qwvxH4DEk4eKT7TfxWC2Re6QEWmqxWXctytfa7): protocol: expected piece hash; context canceled; EOF Aug 12 17:13:31.393705218 (95h47m25.824785339s) [noise=true , address=79.127.219.51:4195 , piece_id=4EU53SFRSCZ2NNZBFWI427YQHR7BEFT5JX6TW7E33P2TRIN4FQXQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mFXuncCmi6nsnuVqGK9QxfGCzGXgX3kcPPy9icg1fPemHpsx): protocol: expected piece hash; context canceled; EOF Aug 12 17:13:31.393756705 (95h47m25.824836836s) [noise=true , address=104.36.82.62:4441 , piece_id=MC7SVGPDMN4VVKEOB5CEERHLNI53KA6XGSWXE6OQWCMEL2XTWVJA, seg_pos={0 0}, upload=99787]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VBayWisMK1n3cwypeiLzGuffonBTzYga7thAZ8pSqrusc7kow): protocol: expected piece hash; context canceled; EOF Aug 12 17:13:31.393778618 (95h47m25.824858739s) [noise=true , address=104.36.82.7:4915 , piece_id=35GNLEYYZL7J2RXDJ6ONZOX3SHEJHUCKVJDAGV64KWKURW5PCTNQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt): protocol: expected piece hash; context canceled; EOF Aug 12 17:13:31.393833052 (95h47m25.824913183s) [seg_pos={0 0}, upload=99787]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 12 17:13:31.393916311 (95h47m25.824996442s) [upload=99787]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 17:13:31.393920689 (95h47m25.82500082s) [upload=99787]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 17:13:31.701564897 (95h47m26.132645038s) [upload=99787]: Done uploading segments... Aug 12 17:13:31.701588933 (95h47m26.132669114s) [upload=99787]: Upload finished. err: <nil> Duration: 5.956926197s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 6.058919803s FAIL (2025-08-12 17:15:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 12 16:51:00.342944596 (95h24m54.774024727s) [upload=99416]: Starting upload Aug 12 16:51:00.343003058 (95h24m54.774083189s) [upload=99416]: Done starting upload Aug 12 16:51:00.343093682 (95h24m54.774173803s) [upload=99416]: Uploading segments... Aug 12 16:51:00.343214823 (95h24m54.774294954s) [upload=99416]: Got next segment. Inline: false Aug 12 16:51:00.343269027 (95h24m54.774349158s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:51:00.343273756 (95h24m54.774353887s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 16:51:00.585349151 (95h24m55.016429282s) [seg_pos={0 0}, upload=99416]: Begin upload segment... Aug 12 16:51:00.585396822 (95h24m55.016476973s) [seg_pos={0 0}, upload=99416]: Done begin upload segment. Aug 12 16:51:00.585410067 (95h24m55.016490208s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585507734 (95h24m55.016587895s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585886159 (95h24m55.0169663s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586089607 (95h24m55.017169738s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586541652 (95h24m55.017621803s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587004688 (95h24m55.018084819s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587619234 (95h24m55.018699385s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587983842 (95h24m55.019063983s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588303153 (95h24m55.019383284s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588688090 (95h24m55.019768221s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.589902393 (95h24m55.020982534s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.590674450 (95h24m55.021754601s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.591296961 (95h24m55.022377092s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.592189969 (95h24m55.02327011s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.593156619 (95h24m55.02423678s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.594056290 (95h24m55.025136421s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:02.362133336 (95h24m56.793213467s) [upload=99416]: Next returned nil segment Aug 12 16:51:02.362159547 (95h24m56.793239678s) [upload=99416]: Waiting for error group managing segments... Aug 12 16:51:02.650293798 (95h24m57.081373929s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.663772758 (95h24m57.094852919s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.696344749 (95h24m57.12742488s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.709104884 (95h24m57.140185025s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.722233484 (95h24m57.153313625s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.727048907 (95h24m57.158129038s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.741870376 (95h24m57.172950507s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.772732365 (95h24m57.203812507s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.805755481 (95h24m57.236835612s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.878280266 (95h24m57.309360397s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.913734542 (95h24m57.344814673s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.972369120 (95h24m57.403449251s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983815812 (95h24m57.414895943s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983842733 (95h24m57.414922864s) [seg_pos={0 0}, upload=99416]: Segment reached optimal threshold of 13 pieces. Aug 12 16:51:02.983942084 (95h24m57.415022215s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.983999583 (95h24m57.415079715s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984087211 (95h24m57.415167342s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984124363 (95h24m57.415204494s) [seg_pos={0 0}, upload=99416]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 12 16:51:02.984229683 (95h24m57.415309814s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 16:51:02.984234323 (95h24m57.415314454s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:51:03.289516404 (95h24m57.720596575s) [upload=99416]: Done uploading segments... Aug 12 16:51:03.289540049 (95h24m57.7206202s) [upload=99416]: Upload finished. err: <nil> Duration: 6.058948979s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.997408261s FAIL (2025-08-12 17:15:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 12 16:51:00.342944596 (95h24m54.774024727s) [upload=99416]: Starting upload Aug 12 16:51:00.343003058 (95h24m54.774083189s) [upload=99416]: Done starting upload Aug 12 16:51:00.343093682 (95h24m54.774173803s) [upload=99416]: Uploading segments... Aug 12 16:51:00.343214823 (95h24m54.774294954s) [upload=99416]: Got next segment. Inline: false Aug 12 16:51:00.343269027 (95h24m54.774349158s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:51:00.343273756 (95h24m54.774353887s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 16:51:00.585349151 (95h24m55.016429282s) [seg_pos={0 0}, upload=99416]: Begin upload segment... Aug 12 16:51:00.585396822 (95h24m55.016476973s) [seg_pos={0 0}, upload=99416]: Done begin upload segment. Aug 12 16:51:00.585410067 (95h24m55.016490208s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585507734 (95h24m55.016587895s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585886159 (95h24m55.0169663s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586089607 (95h24m55.017169738s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586541652 (95h24m55.017621803s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587004688 (95h24m55.018084819s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587619234 (95h24m55.018699385s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587983842 (95h24m55.019063983s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588303153 (95h24m55.019383284s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588688090 (95h24m55.019768221s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.589902393 (95h24m55.020982534s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.590674450 (95h24m55.021754601s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.591296961 (95h24m55.022377092s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.592189969 (95h24m55.02327011s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.593156619 (95h24m55.02423678s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.594056290 (95h24m55.025136421s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:02.362133336 (95h24m56.793213467s) [upload=99416]: Next returned nil segment Aug 12 16:51:02.362159547 (95h24m56.793239678s) [upload=99416]: Waiting for error group managing segments... Aug 12 16:51:02.650293798 (95h24m57.081373929s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.663772758 (95h24m57.094852919s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.696344749 (95h24m57.12742488s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.709104884 (95h24m57.140185025s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.722233484 (95h24m57.153313625s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.727048907 (95h24m57.158129038s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.741870376 (95h24m57.172950507s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.772732365 (95h24m57.203812507s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.805755481 (95h24m57.236835612s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.878280266 (95h24m57.309360397s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.913734542 (95h24m57.344814673s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.972369120 (95h24m57.403449251s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983815812 (95h24m57.414895943s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983842733 (95h24m57.414922864s) [seg_pos={0 0}, upload=99416]: Segment reached optimal threshold of 13 pieces. Aug 12 16:51:02.983942084 (95h24m57.415022215s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.983999583 (95h24m57.415079715s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984087211 (95h24m57.415167342s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984124363 (95h24m57.415204494s) [seg_pos={0 0}, upload=99416]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 12 16:51:02.984229683 (95h24m57.415309814s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 16:51:02.984234323 (95h24m57.415314454s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:51:03.289516404 (95h24m57.720596575s) [upload=99416]: Done uploading segments... Aug 12 16:51:03.289540049 (95h24m57.7206202s) [upload=99416]: Upload finished. err: <nil> Duration: 5.997441945s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...X..........X...........................................X.] duration: 1.924824053s FAIL (2025-08-12 17:14:45 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 12 17:14:10.610708770 (95h48m5.041788891s) [upload=99796]: Starting upload Aug 12 17:14:10.610760419 (95h48m5.04184055s) [upload=99796]: Done starting upload Aug 12 17:14:10.610837096 (95h48m5.041917227s) [upload=99796]: Uploading segments... Aug 12 17:14:10.610918822 (95h48m5.041998954s) [upload=99796]: Got next segment. Inline: true Aug 12 17:14:10.610924463 (95h48m5.042004594s) [upload=99796]: Waiting for error group managing segments... Aug 12 17:14:10.610977775 (95h48m5.042057906s) [upload=99796]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 17:14:10.610983366 (95h48m5.042063497s) [upload=99796]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 17:14:10.610985930 (95h48m5.042066051s) [upload=99796]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 17:14:10.896688638 (95h48m5.327768759s) [upload=99796]: Done uploading segments... Aug 12 17:14:10.896709046 (95h48m5.327789177s) [upload=99796]: Upload finished. err: <nil> Duration:* 1.92484912s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.............................X.X.........X..........X.....X.] duration: 6.006944264s FAIL (2025-08-12 17:14:44 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 12 17:13:28.825252727 (95h47m23.256332898s) [upload=99787]: Starting upload Aug 12 17:13:28.825321419 (95h47m23.25640159s) [upload=99787]: Done starting upload Aug 12 17:13:28.825449604 (95h47m23.256529775s) [upload=99787]: Uploading segments... Aug 12 17:13:28.825744037 (95h47m23.256824198s) [upload=99787]: Got next segment. Inline: false Aug 12 17:13:28.825821235 (95h47m23.256901416s) [upload=99787]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 17:13:28.825829931 (95h47m23.256910093s) [upload=99787]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 17:13:29.303728856 (95h47m23.734808987s) [seg_pos={0 0}, upload=99787]: Begin upload segment... Aug 12 17:13:29.303773140 (95h47m23.734853271s) [seg_pos={0 0}, upload=99787]: Done begin upload segment. Aug 12 17:13:29.303793930 (95h47m23.734874061s) [noise=true , address=104.36.82.44:4900 , piece_id=LCDUXPUFOZ43NKZ4LOR5ZCVQ3RPPCTIR2UU7XDNNFHHXDS77UKFQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.303866910 (95h47m23.734947041s) [noise=true , address=104.36.82.58:4266 , piece_id=XOTBH7GZGN7QZFS2U2AS72TKR3AU2KYQD474Y6GMFFUYT4LHPLHQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.304176021 (95h47m23.735256152s) [noise=true , address=104.36.82.38:4694 , piece_id=RZ6LNKXDHDRU73N6NIULQG34PVWCYWIVA4FJ2BEF3CKUCFXGAKMQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.304419978 (95h47m23.735500109s) [noise=true , address=104.36.82.9:4025 , piece_id=AQJMXPICHPUI5UPVL6WZWMGZ7UFBVDK5KS4IFV3XBOXX76DMJVRA, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.304639759 (95h47m23.73571988s) [noise=true , address=79.127.219.51:4195 , piece_id=4EU53SFRSCZ2NNZBFWI427YQHR7BEFT5JX6TW7E33P2TRIN4FQXQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.304780888 (95h47m23.735861019s) [noise=true , address=64.49.13.154:4936 , piece_id=FF3TYKME6R42QWQEW27PSKJ5ORUC4A7SMWVHPH6L2ODX3TISQH5A, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.304926106 (95h47m23.736006227s) [noise=true , address=104.36.82.62:4441 , piece_id=MC7SVGPDMN4VVKEOB5CEERHLNI53KA6XGSWXE6OQWCMEL2XTWVJA, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.305191725 (95h47m23.736271856s) [noise=true , address=104.36.82.56:4237 , piece_id=BV33QX3IWIUX45JH7NJPW6A2BYF2CWF3PYVPC4YJQ75KCZ3JEXVA, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.305257680 (95h47m23.736337821s) [noise=true , address=64.49.13.140:4506 , piece_id=SSZ5XSTLM4CKZTRLABB2CC5BK54E3BGYX5HWWLXRNVK5BIPRVEFQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.305590016 (95h47m23.736670147s) [noise=true , address=104.36.82.19:4402 , piece_id=IEL4AM22HO26IC5TSAFKXCMHY3B4MGPOTDD2HVQPYEFD3GCJCADQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.305886914 (95h47m23.736967035s) [noise=true , address=104.36.82.17:4345 , piece_id=VVRYNEI7FMSK2QHNR7N2EACZQZVBKG54ADWESJNGS43PVEG5EMLQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.305961698 (95h47m23.737041829s) [noise=true , address=104.36.82.45:4931 , piece_id=WKX4RE7H2PMKMEZWTKT6EF2BALALCT2ZPNFAUFMC7IFUJITEJ7AQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306106645 (95h47m23.737186776s) [noise=true , address=104.36.82.7:4915 , piece_id=35GNLEYYZL7J2RXDJ6ONZOX3SHEJHUCKVJDAGV64KWKURW5PCTNQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306212778 (95h47m23.737292909s) [noise=true , address=104.36.82.11:4142 , piece_id=7D5FHBGX6S6CTMTK6IEUEYNOT73LQXJYZV6HMUVVP6OOPNR46R6Q, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306306056 (95h47m23.737386187s) [noise=true , address=104.36.82.39:4736 , piece_id=77BJAQPGH6SJN3D3K5UPCA2DNG4T7SK5ECK6ZN2LSVK46EYRXAEQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306459700 (95h47m23.737539832s) [noise=true , address=104.36.82.17:4344 , piece_id=PN5TP33O3LL7W5QJKB24GTSPNXBBKXIGETVIHSBRG4O252NY3T3A, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306513974 (95h47m23.737594105s) [noise=true , address=104.36.82.60:4348 , piece_id=ARK7NJB5532ZZHNVMVYG3XCBOUT4JCO5CEBRC2365T66ERZSTJEA, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306714267 (95h47m23.737794398s) [noise=true , address=77.90.31.16:4764 , piece_id=PVK7KVJD4ZTS2EA2FOU5APWROVZNYCZHGN3M4BXRHU4QJXGDGWWA, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:30.465351213 (95h47m24.896431344s) [upload=99787]: Next returned nil segment Aug 12 17:13:30.465368435 (95h47m24.896448567s) [upload=99787]: Waiting for error group managing segments... Aug 12 17:13:30.784923783 (95h47m25.216003914s) [noise=true , address=64.49.13.140:4506 , piece_id=SSZ5XSTLM4CKZTRLABB2CC5BK54E3BGYX5HWWLXRNVK5BIPRVEFQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:30.790033850 (95h47m25.221113981s) [noise=true , address=64.49.13.154:4936 , piece_id=FF3TYKME6R42QWQEW27PSKJ5ORUC4A7SMWVHPH6L2ODX3TISQH5A, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.255533719 (95h47m25.68661386s) [noise=true , address=104.36.82.38:4694 , piece_id=RZ6LNKXDHDRU73N6NIULQG34PVWCYWIVA4FJ2BEF3CKUCFXGAKMQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.269997042 (95h47m25.701077183s) [noise=true , address=104.36.82.60:4348 , piece_id=ARK7NJB5532ZZHNVMVYG3XCBOUT4JCO5CEBRC2365T66ERZSTJEA, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.302029471 (95h47m25.733109622s) [noise=true , address=77.90.31.16:4764 , piece_id=PVK7KVJD4ZTS2EA2FOU5APWROVZNYCZHGN3M4BXRHU4QJXGDGWWA, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.318353594 (95h47m25.749433725s) [noise=true , address=104.36.82.19:4402 , piece_id=IEL4AM22HO26IC5TSAFKXCMHY3B4MGPOTDD2HVQPYEFD3GCJCADQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.346017534 (95h47m25.777097665s) [noise=true , address=104.36.82.9:4025 , piece_id=AQJMXPICHPUI5UPVL6WZWMGZ7UFBVDK5KS4IFV3XBOXX76DMJVRA, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.348171306 (95h47m25.779251437s) [noise=true , address=104.36.82.45:4931 , piece_id=WKX4RE7H2PMKMEZWTKT6EF2BALALCT2ZPNFAUFMC7IFUJITEJ7AQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.354598163 (95h47m25.785678294s) [noise=true , address=104.36.82.56:4237 , piece_id=BV33QX3IWIUX45JH7NJPW6A2BYF2CWF3PYVPC4YJQ75KCZ3JEXVA, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.360073037 (95h47m25.791153169s) [noise=true , address=104.36.82.17:4345 , piece_id=VVRYNEI7FMSK2QHNR7N2EACZQZVBKG54ADWESJNGS43PVEG5EMLQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.371327111 (95h47m25.802407242s) [noise=true , address=104.36.82.39:4736 , piece_id=77BJAQPGH6SJN3D3K5UPCA2DNG4T7SK5ECK6ZN2LSVK46EYRXAEQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.372622479 (95h47m25.8037026s) [noise=true , address=104.36.82.44:4900 , piece_id=LCDUXPUFOZ43NKZ4LOR5ZCVQ3RPPCTIR2UU7XDNNFHHXDS77UKFQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.379217397 (95h47m25.810297528s) [noise=true , address=104.36.82.17:4344 , piece_id=PN5TP33O3LL7W5QJKB24GTSPNXBBKXIGETVIHSBRG4O252NY3T3A, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.393498702 (95h47m25.824578823s) [noise=true , address=104.36.82.58:4266 , piece_id=XOTBH7GZGN7QZFS2U2AS72TKR3AU2KYQD474Y6GMFFUYT4LHPLHQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.393527026 (95h47m25.824607157s) [seg_pos={0 0}, upload=99787]: Segment reached optimal threshold of 14 pieces. Aug 12 17:13:31.393647137 (95h47m25.824727268s) [noise=true , address=104.36.82.11:4142 , piece_id=7D5FHBGX6S6CTMTK6IEUEYNOT73LQXJYZV6HMUVVP6OOPNR46R6Q, seg_pos={0 0}, upload=99787]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JR1vFqWgTH1qwvxH4DEk4eKT7TfxWC2Re6QEWmqxWXctytfa7): protocol: expected piece hash; context canceled; EOF Aug 12 17:13:31.393705218 (95h47m25.824785339s) [noise=true , address=79.127.219.51:4195 , piece_id=4EU53SFRSCZ2NNZBFWI427YQHR7BEFT5JX6TW7E33P2TRIN4FQXQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mFXuncCmi6nsnuVqGK9QxfGCzGXgX3kcPPy9icg1fPemHpsx): protocol: expected piece hash; context canceled; EOF Aug 12 17:13:31.393756705 (95h47m25.824836836s) [noise=true , address=104.36.82.62:4441 , piece_id=MC7SVGPDMN4VVKEOB5CEERHLNI53KA6XGSWXE6OQWCMEL2XTWVJA, seg_pos={0 0}, upload=99787]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VBayWisMK1n3cwypeiLzGuffonBTzYga7thAZ8pSqrusc7kow): protocol: expected piece hash; context canceled; EOF Aug 12 17:13:31.393778618 (95h47m25.824858739s) [noise=true , address=104.36.82.7:4915 , piece_id=35GNLEYYZL7J2RXDJ6ONZOX3SHEJHUCKVJDAGV64KWKURW5PCTNQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt): protocol: expected piece hash; context canceled; EOF Aug 12 17:13:31.393833052 (95h47m25.824913183s) [seg_pos={0 0}, upload=99787]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 12 17:13:31.393916311 (95h47m25.824996442s) [upload=99787]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 17:13:31.393920689 (95h47m25.82500082s) [upload=99787]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 17:13:31.701564897 (95h47m26.132645038s) [upload=99787]: Done uploading segments... Aug 12 17:13:31.701588933 (95h47m26.132669114s) [upload=99787]: Upload finished. err: <nil> Duration: 6.006983409s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.027140046s FAIL (2025-08-12 17:14:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 12 16:51:00.342944596 (95h24m54.774024727s) [upload=99416]: Starting upload Aug 12 16:51:00.343003058 (95h24m54.774083189s) [upload=99416]: Done starting upload Aug 12 16:51:00.343093682 (95h24m54.774173803s) [upload=99416]: Uploading segments... Aug 12 16:51:00.343214823 (95h24m54.774294954s) [upload=99416]: Got next segment. Inline: false Aug 12 16:51:00.343269027 (95h24m54.774349158s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:51:00.343273756 (95h24m54.774353887s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 16:51:00.585349151 (95h24m55.016429282s) [seg_pos={0 0}, upload=99416]: Begin upload segment... Aug 12 16:51:00.585396822 (95h24m55.016476973s) [seg_pos={0 0}, upload=99416]: Done begin upload segment. Aug 12 16:51:00.585410067 (95h24m55.016490208s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585507734 (95h24m55.016587895s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585886159 (95h24m55.0169663s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586089607 (95h24m55.017169738s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586541652 (95h24m55.017621803s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587004688 (95h24m55.018084819s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587619234 (95h24m55.018699385s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587983842 (95h24m55.019063983s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588303153 (95h24m55.019383284s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588688090 (95h24m55.019768221s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.589902393 (95h24m55.020982534s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.590674450 (95h24m55.021754601s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.591296961 (95h24m55.022377092s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.592189969 (95h24m55.02327011s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.593156619 (95h24m55.02423678s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.594056290 (95h24m55.025136421s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:02.362133336 (95h24m56.793213467s) [upload=99416]: Next returned nil segment Aug 12 16:51:02.362159547 (95h24m56.793239678s) [upload=99416]: Waiting for error group managing segments... Aug 12 16:51:02.650293798 (95h24m57.081373929s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.663772758 (95h24m57.094852919s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.696344749 (95h24m57.12742488s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.709104884 (95h24m57.140185025s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.722233484 (95h24m57.153313625s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.727048907 (95h24m57.158129038s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.741870376 (95h24m57.172950507s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.772732365 (95h24m57.203812507s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.805755481 (95h24m57.236835612s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.878280266 (95h24m57.309360397s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.913734542 (95h24m57.344814673s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.972369120 (95h24m57.403449251s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983815812 (95h24m57.414895943s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983842733 (95h24m57.414922864s) [seg_pos={0 0}, upload=99416]: Segment reached optimal threshold of 13 pieces. Aug 12 16:51:02.983942084 (95h24m57.415022215s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.983999583 (95h24m57.415079715s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984087211 (95h24m57.415167342s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984124363 (95h24m57.415204494s) [seg_pos={0 0}, upload=99416]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 12 16:51:02.984229683 (95h24m57.415309814s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 16:51:02.984234323 (95h24m57.415314454s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:51:03.289516404 (95h24m57.720596575s) [upload=99416]: Done uploading segments... Aug 12 16:51:03.289540049 (95h24m57.7206202s) [upload=99416]: Upload finished. err: <nil> Duration: 6.02716302s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [....X..........X...........................................X] duration: 1.621268644s FAIL (2025-08-12 17:14:11 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 12 17:14:10.610708770 (95h48m5.041788891s) [upload=99796]: Starting upload Aug 12 17:14:10.610760419 (95h48m5.04184055s) [upload=99796]: Done starting upload Aug 12 17:14:10.610837096 (95h48m5.041917227s) [upload=99796]: Uploading segments... Aug 12 17:14:10.610918822 (95h48m5.041998954s) [upload=99796]: Got next segment. Inline: true Aug 12 17:14:10.610924463 (95h48m5.042004594s) [upload=99796]: Waiting for error group managing segments... Aug 12 17:14:10.610977775 (95h48m5.042057906s) [upload=99796]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 17:14:10.610983366 (95h48m5.042063497s) [upload=99796]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 17:14:10.610985930 (95h48m5.042066051s) [upload=99796]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 17:14:10.896688638 (95h48m5.327768759s) [upload=99796]: Done uploading segments... Aug 12 17:14:10.896709046 (95h48m5.327789177s) [upload=99796]: Upload finished. err: <nil> Duration:* 1.621302069s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.873670601s FAIL (2025-08-12 17:14:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 12 16:51:00.342944596 (95h24m54.774024727s) [upload=99416]: Starting upload Aug 12 16:51:00.343003058 (95h24m54.774083189s) [upload=99416]: Done starting upload Aug 12 16:51:00.343093682 (95h24m54.774173803s) [upload=99416]: Uploading segments... Aug 12 16:51:00.343214823 (95h24m54.774294954s) [upload=99416]: Got next segment. Inline: false Aug 12 16:51:00.343269027 (95h24m54.774349158s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:51:00.343273756 (95h24m54.774353887s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 16:51:00.585349151 (95h24m55.016429282s) [seg_pos={0 0}, upload=99416]: Begin upload segment... Aug 12 16:51:00.585396822 (95h24m55.016476973s) [seg_pos={0 0}, upload=99416]: Done begin upload segment. Aug 12 16:51:00.585410067 (95h24m55.016490208s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585507734 (95h24m55.016587895s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585886159 (95h24m55.0169663s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586089607 (95h24m55.017169738s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586541652 (95h24m55.017621803s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587004688 (95h24m55.018084819s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587619234 (95h24m55.018699385s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587983842 (95h24m55.019063983s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588303153 (95h24m55.019383284s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588688090 (95h24m55.019768221s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.589902393 (95h24m55.020982534s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.590674450 (95h24m55.021754601s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.591296961 (95h24m55.022377092s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.592189969 (95h24m55.02327011s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.593156619 (95h24m55.02423678s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.594056290 (95h24m55.025136421s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:02.362133336 (95h24m56.793213467s) [upload=99416]: Next returned nil segment Aug 12 16:51:02.362159547 (95h24m56.793239678s) [upload=99416]: Waiting for error group managing segments... Aug 12 16:51:02.650293798 (95h24m57.081373929s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.663772758 (95h24m57.094852919s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.696344749 (95h24m57.12742488s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.709104884 (95h24m57.140185025s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.722233484 (95h24m57.153313625s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.727048907 (95h24m57.158129038s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.741870376 (95h24m57.172950507s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.772732365 (95h24m57.203812507s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.805755481 (95h24m57.236835612s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.878280266 (95h24m57.309360397s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.913734542 (95h24m57.344814673s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.972369120 (95h24m57.403449251s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983815812 (95h24m57.414895943s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983842733 (95h24m57.414922864s) [seg_pos={0 0}, upload=99416]: Segment reached optimal threshold of 13 pieces. Aug 12 16:51:02.983942084 (95h24m57.415022215s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.983999583 (95h24m57.415079715s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984087211 (95h24m57.415167342s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984124363 (95h24m57.415204494s) [seg_pos={0 0}, upload=99416]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 12 16:51:02.984229683 (95h24m57.415309814s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 16:51:02.984234323 (95h24m57.415314454s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:51:03.289516404 (95h24m57.720596575s) [upload=99416]: Done uploading segments... Aug 12 16:51:03.289540049 (95h24m57.7206202s) [upload=99416]: Upload finished. err: <nil> Duration: 5.87369656s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [X.............................X.X.........X..........X.....X] duration: 4.210764708s FAIL (2025-08-12 17:13:32 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 12 17:13:28.825252727 (95h47m23.256332898s) [upload=99787]: Starting upload Aug 12 17:13:28.825321419 (95h47m23.25640159s) [upload=99787]: Done starting upload Aug 12 17:13:28.825449604 (95h47m23.256529775s) [upload=99787]: Uploading segments... Aug 12 17:13:28.825744037 (95h47m23.256824198s) [upload=99787]: Got next segment. Inline: false Aug 12 17:13:28.825821235 (95h47m23.256901416s) [upload=99787]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 17:13:28.825829931 (95h47m23.256910093s) [upload=99787]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 17:13:29.303728856 (95h47m23.734808987s) [seg_pos={0 0}, upload=99787]: Begin upload segment... Aug 12 17:13:29.303773140 (95h47m23.734853271s) [seg_pos={0 0}, upload=99787]: Done begin upload segment. Aug 12 17:13:29.303793930 (95h47m23.734874061s) [noise=true , address=104.36.82.44:4900 , piece_id=LCDUXPUFOZ43NKZ4LOR5ZCVQ3RPPCTIR2UU7XDNNFHHXDS77UKFQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.303866910 (95h47m23.734947041s) [noise=true , address=104.36.82.58:4266 , piece_id=XOTBH7GZGN7QZFS2U2AS72TKR3AU2KYQD474Y6GMFFUYT4LHPLHQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.304176021 (95h47m23.735256152s) [noise=true , address=104.36.82.38:4694 , piece_id=RZ6LNKXDHDRU73N6NIULQG34PVWCYWIVA4FJ2BEF3CKUCFXGAKMQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.304419978 (95h47m23.735500109s) [noise=true , address=104.36.82.9:4025 , piece_id=AQJMXPICHPUI5UPVL6WZWMGZ7UFBVDK5KS4IFV3XBOXX76DMJVRA, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.304639759 (95h47m23.73571988s) [noise=true , address=79.127.219.51:4195 , piece_id=4EU53SFRSCZ2NNZBFWI427YQHR7BEFT5JX6TW7E33P2TRIN4FQXQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.304780888 (95h47m23.735861019s) [noise=true , address=64.49.13.154:4936 , piece_id=FF3TYKME6R42QWQEW27PSKJ5ORUC4A7SMWVHPH6L2ODX3TISQH5A, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.304926106 (95h47m23.736006227s) [noise=true , address=104.36.82.62:4441 , piece_id=MC7SVGPDMN4VVKEOB5CEERHLNI53KA6XGSWXE6OQWCMEL2XTWVJA, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.305191725 (95h47m23.736271856s) [noise=true , address=104.36.82.56:4237 , piece_id=BV33QX3IWIUX45JH7NJPW6A2BYF2CWF3PYVPC4YJQ75KCZ3JEXVA, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.305257680 (95h47m23.736337821s) [noise=true , address=64.49.13.140:4506 , piece_id=SSZ5XSTLM4CKZTRLABB2CC5BK54E3BGYX5HWWLXRNVK5BIPRVEFQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.305590016 (95h47m23.736670147s) [noise=true , address=104.36.82.19:4402 , piece_id=IEL4AM22HO26IC5TSAFKXCMHY3B4MGPOTDD2HVQPYEFD3GCJCADQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.305886914 (95h47m23.736967035s) [noise=true , address=104.36.82.17:4345 , piece_id=VVRYNEI7FMSK2QHNR7N2EACZQZVBKG54ADWESJNGS43PVEG5EMLQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.305961698 (95h47m23.737041829s) [noise=true , address=104.36.82.45:4931 , piece_id=WKX4RE7H2PMKMEZWTKT6EF2BALALCT2ZPNFAUFMC7IFUJITEJ7AQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306106645 (95h47m23.737186776s) [noise=true , address=104.36.82.7:4915 , piece_id=35GNLEYYZL7J2RXDJ6ONZOX3SHEJHUCKVJDAGV64KWKURW5PCTNQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306212778 (95h47m23.737292909s) [noise=true , address=104.36.82.11:4142 , piece_id=7D5FHBGX6S6CTMTK6IEUEYNOT73LQXJYZV6HMUVVP6OOPNR46R6Q, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306306056 (95h47m23.737386187s) [noise=true , address=104.36.82.39:4736 , piece_id=77BJAQPGH6SJN3D3K5UPCA2DNG4T7SK5ECK6ZN2LSVK46EYRXAEQ, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306459700 (95h47m23.737539832s) [noise=true , address=104.36.82.17:4344 , piece_id=PN5TP33O3LL7W5QJKB24GTSPNXBBKXIGETVIHSBRG4O252NY3T3A, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306513974 (95h47m23.737594105s) [noise=true , address=104.36.82.60:4348 , piece_id=ARK7NJB5532ZZHNVMVYG3XCBOUT4JCO5CEBRC2365T66ERZSTJEA, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:29.306714267 (95h47m23.737794398s) [noise=true , address=77.90.31.16:4764 , piece_id=PVK7KVJD4ZTS2EA2FOU5APWROVZNYCZHGN3M4BXRHU4QJXGDGWWA, seg_pos={0 0}, upload=99787]: Uploading piece... Aug 12 17:13:30.465351213 (95h47m24.896431344s) [upload=99787]: Next returned nil segment Aug 12 17:13:30.465368435 (95h47m24.896448567s) [upload=99787]: Waiting for error group managing segments... Aug 12 17:13:30.784923783 (95h47m25.216003914s) [noise=true , address=64.49.13.140:4506 , piece_id=SSZ5XSTLM4CKZTRLABB2CC5BK54E3BGYX5HWWLXRNVK5BIPRVEFQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:30.790033850 (95h47m25.221113981s) [noise=true , address=64.49.13.154:4936 , piece_id=FF3TYKME6R42QWQEW27PSKJ5ORUC4A7SMWVHPH6L2ODX3TISQH5A, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.255533719 (95h47m25.68661386s) [noise=true , address=104.36.82.38:4694 , piece_id=RZ6LNKXDHDRU73N6NIULQG34PVWCYWIVA4FJ2BEF3CKUCFXGAKMQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.269997042 (95h47m25.701077183s) [noise=true , address=104.36.82.60:4348 , piece_id=ARK7NJB5532ZZHNVMVYG3XCBOUT4JCO5CEBRC2365T66ERZSTJEA, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.302029471 (95h47m25.733109622s) [noise=true , address=77.90.31.16:4764 , piece_id=PVK7KVJD4ZTS2EA2FOU5APWROVZNYCZHGN3M4BXRHU4QJXGDGWWA, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.318353594 (95h47m25.749433725s) [noise=true , address=104.36.82.19:4402 , piece_id=IEL4AM22HO26IC5TSAFKXCMHY3B4MGPOTDD2HVQPYEFD3GCJCADQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.346017534 (95h47m25.777097665s) [noise=true , address=104.36.82.9:4025 , piece_id=AQJMXPICHPUI5UPVL6WZWMGZ7UFBVDK5KS4IFV3XBOXX76DMJVRA, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.348171306 (95h47m25.779251437s) [noise=true , address=104.36.82.45:4931 , piece_id=WKX4RE7H2PMKMEZWTKT6EF2BALALCT2ZPNFAUFMC7IFUJITEJ7AQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.354598163 (95h47m25.785678294s) [noise=true , address=104.36.82.56:4237 , piece_id=BV33QX3IWIUX45JH7NJPW6A2BYF2CWF3PYVPC4YJQ75KCZ3JEXVA, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.360073037 (95h47m25.791153169s) [noise=true , address=104.36.82.17:4345 , piece_id=VVRYNEI7FMSK2QHNR7N2EACZQZVBKG54ADWESJNGS43PVEG5EMLQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.371327111 (95h47m25.802407242s) [noise=true , address=104.36.82.39:4736 , piece_id=77BJAQPGH6SJN3D3K5UPCA2DNG4T7SK5ECK6ZN2LSVK46EYRXAEQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.372622479 (95h47m25.8037026s) [noise=true , address=104.36.82.44:4900 , piece_id=LCDUXPUFOZ43NKZ4LOR5ZCVQ3RPPCTIR2UU7XDNNFHHXDS77UKFQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.379217397 (95h47m25.810297528s) [noise=true , address=104.36.82.17:4344 , piece_id=PN5TP33O3LL7W5QJKB24GTSPNXBBKXIGETVIHSBRG4O252NY3T3A, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.393498702 (95h47m25.824578823s) [noise=true , address=104.36.82.58:4266 , piece_id=XOTBH7GZGN7QZFS2U2AS72TKR3AU2KYQD474Y6GMFFUYT4LHPLHQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: <nil> Aug 12 17:13:31.393527026 (95h47m25.824607157s) [seg_pos={0 0}, upload=99787]: Segment reached optimal threshold of 14 pieces. Aug 12 17:13:31.393647137 (95h47m25.824727268s) [noise=true , address=104.36.82.11:4142 , piece_id=7D5FHBGX6S6CTMTK6IEUEYNOT73LQXJYZV6HMUVVP6OOPNR46R6Q, seg_pos={0 0}, upload=99787]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JR1vFqWgTH1qwvxH4DEk4eKT7TfxWC2Re6QEWmqxWXctytfa7): protocol: expected piece hash; context canceled; EOF Aug 12 17:13:31.393705218 (95h47m25.824785339s) [noise=true , address=79.127.219.51:4195 , piece_id=4EU53SFRSCZ2NNZBFWI427YQHR7BEFT5JX6TW7E33P2TRIN4FQXQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mFXuncCmi6nsnuVqGK9QxfGCzGXgX3kcPPy9icg1fPemHpsx): protocol: expected piece hash; context canceled; EOF Aug 12 17:13:31.393756705 (95h47m25.824836836s) [noise=true , address=104.36.82.62:4441 , piece_id=MC7SVGPDMN4VVKEOB5CEERHLNI53KA6XGSWXE6OQWCMEL2XTWVJA, seg_pos={0 0}, upload=99787]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VBayWisMK1n3cwypeiLzGuffonBTzYga7thAZ8pSqrusc7kow): protocol: expected piece hash; context canceled; EOF Aug 12 17:13:31.393778618 (95h47m25.824858739s) [noise=true , address=104.36.82.7:4915 , piece_id=35GNLEYYZL7J2RXDJ6ONZOX3SHEJHUCKVJDAGV64KWKURW5PCTNQ, seg_pos={0 0}, upload=99787]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt): protocol: expected piece hash; context canceled; EOF Aug 12 17:13:31.393833052 (95h47m25.824913183s) [seg_pos={0 0}, upload=99787]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 12 17:13:31.393916311 (95h47m25.824996442s) [upload=99787]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 17:13:31.393920689 (95h47m25.82500082s) [upload=99787]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 17:13:31.701564897 (95h47m26.132645038s) [upload=99787]: Done uploading segments... Aug 12 17:13:31.701588933 (95h47m26.132669114s) [upload=99787]: Upload finished. err: <nil> Duration: 4.210791921s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.030138033s FAIL (2025-08-12 17:13:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 12 16:51:00.342944596 (95h24m54.774024727s) [upload=99416]: Starting upload Aug 12 16:51:00.343003058 (95h24m54.774083189s) [upload=99416]: Done starting upload Aug 12 16:51:00.343093682 (95h24m54.774173803s) [upload=99416]: Uploading segments... Aug 12 16:51:00.343214823 (95h24m54.774294954s) [upload=99416]: Got next segment. Inline: false Aug 12 16:51:00.343269027 (95h24m54.774349158s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:51:00.343273756 (95h24m54.774353887s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 16:51:00.585349151 (95h24m55.016429282s) [seg_pos={0 0}, upload=99416]: Begin upload segment... Aug 12 16:51:00.585396822 (95h24m55.016476973s) [seg_pos={0 0}, upload=99416]: Done begin upload segment. Aug 12 16:51:00.585410067 (95h24m55.016490208s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585507734 (95h24m55.016587895s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585886159 (95h24m55.0169663s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586089607 (95h24m55.017169738s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586541652 (95h24m55.017621803s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587004688 (95h24m55.018084819s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587619234 (95h24m55.018699385s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587983842 (95h24m55.019063983s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588303153 (95h24m55.019383284s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588688090 (95h24m55.019768221s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.589902393 (95h24m55.020982534s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.590674450 (95h24m55.021754601s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.591296961 (95h24m55.022377092s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.592189969 (95h24m55.02327011s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.593156619 (95h24m55.02423678s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.594056290 (95h24m55.025136421s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:02.362133336 (95h24m56.793213467s) [upload=99416]: Next returned nil segment Aug 12 16:51:02.362159547 (95h24m56.793239678s) [upload=99416]: Waiting for error group managing segments... Aug 12 16:51:02.650293798 (95h24m57.081373929s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.663772758 (95h24m57.094852919s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.696344749 (95h24m57.12742488s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.709104884 (95h24m57.140185025s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.722233484 (95h24m57.153313625s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.727048907 (95h24m57.158129038s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.741870376 (95h24m57.172950507s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.772732365 (95h24m57.203812507s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.805755481 (95h24m57.236835612s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.878280266 (95h24m57.309360397s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.913734542 (95h24m57.344814673s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.972369120 (95h24m57.403449251s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983815812 (95h24m57.414895943s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983842733 (95h24m57.414922864s) [seg_pos={0 0}, upload=99416]: Segment reached optimal threshold of 13 pieces. Aug 12 16:51:02.983942084 (95h24m57.415022215s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.983999583 (95h24m57.415079715s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984087211 (95h24m57.415167342s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984124363 (95h24m57.415204494s) [seg_pos={0 0}, upload=99416]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 12 16:51:02.984229683 (95h24m57.415309814s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 16:51:02.984234323 (95h24m57.415314454s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:51:03.289516404 (95h24m57.720596575s) [upload=99416]: Done uploading segments... Aug 12 16:51:03.289540049 (95h24m57.7206202s) [upload=99416]: Upload finished. err: <nil> Duration: 6.030181096s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....X..........X...........................................] duration: 1.852118038s FAIL (2025-08-12 17:13:25 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 12 16:39:09.008146755 (95h13m3.439226887s) [upload=99194]: Starting upload Aug 12 16:39:09.008201260 (95h13m3.439281391s) [upload=99194]: Done starting upload Aug 12 16:39:09.008258329 (95h13m3.43933846s) [upload=99194]: Uploading segments... Aug 12 16:39:09.008290200 (95h13m3.439370331s) [upload=99194]: Got next segment. Inline: true Aug 12 16:39:09.008295661 (95h13m3.439375782s) [upload=99194]: Waiting for error group managing segments... Aug 12 16:39:09.008328123 (95h13m3.439408254s) [upload=99194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:39:09.008334585 (95h13m3.439414716s) [upload=99194]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 16:39:09.008337160 (95h13m3.439417281s) [upload=99194]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:39:09.337170261 (95h13m3.768250392s) [upload=99194]: Done uploading segments... Aug 12 16:39:09.337190520 (95h13m3.768270651s) [upload=99194]: Upload finished. err: <nil> Duration:* 1.852142555s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.983825683s FAIL (2025-08-12 17:12:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 12 16:51:00.342944596 (95h24m54.774024727s) [upload=99416]: Starting upload Aug 12 16:51:00.343003058 (95h24m54.774083189s) [upload=99416]: Done starting upload Aug 12 16:51:00.343093682 (95h24m54.774173803s) [upload=99416]: Uploading segments... Aug 12 16:51:00.343214823 (95h24m54.774294954s) [upload=99416]: Got next segment. Inline: false Aug 12 16:51:00.343269027 (95h24m54.774349158s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:51:00.343273756 (95h24m54.774353887s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 16:51:00.585349151 (95h24m55.016429282s) [seg_pos={0 0}, upload=99416]: Begin upload segment... Aug 12 16:51:00.585396822 (95h24m55.016476973s) [seg_pos={0 0}, upload=99416]: Done begin upload segment. Aug 12 16:51:00.585410067 (95h24m55.016490208s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585507734 (95h24m55.016587895s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585886159 (95h24m55.0169663s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586089607 (95h24m55.017169738s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586541652 (95h24m55.017621803s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587004688 (95h24m55.018084819s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587619234 (95h24m55.018699385s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587983842 (95h24m55.019063983s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588303153 (95h24m55.019383284s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588688090 (95h24m55.019768221s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.589902393 (95h24m55.020982534s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.590674450 (95h24m55.021754601s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.591296961 (95h24m55.022377092s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.592189969 (95h24m55.02327011s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.593156619 (95h24m55.02423678s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.594056290 (95h24m55.025136421s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:02.362133336 (95h24m56.793213467s) [upload=99416]: Next returned nil segment Aug 12 16:51:02.362159547 (95h24m56.793239678s) [upload=99416]: Waiting for error group managing segments... Aug 12 16:51:02.650293798 (95h24m57.081373929s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.663772758 (95h24m57.094852919s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.696344749 (95h24m57.12742488s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.709104884 (95h24m57.140185025s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.722233484 (95h24m57.153313625s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.727048907 (95h24m57.158129038s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.741870376 (95h24m57.172950507s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.772732365 (95h24m57.203812507s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.805755481 (95h24m57.236835612s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.878280266 (95h24m57.309360397s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.913734542 (95h24m57.344814673s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.972369120 (95h24m57.403449251s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983815812 (95h24m57.414895943s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983842733 (95h24m57.414922864s) [seg_pos={0 0}, upload=99416]: Segment reached optimal threshold of 13 pieces. Aug 12 16:51:02.983942084 (95h24m57.415022215s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.983999583 (95h24m57.415079715s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984087211 (95h24m57.415167342s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984124363 (95h24m57.415204494s) [seg_pos={0 0}, upload=99416]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 12 16:51:02.984229683 (95h24m57.415309814s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 16:51:02.984234323 (95h24m57.415314454s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:51:03.289516404 (95h24m57.720596575s) [upload=99416]: Done uploading segments... Aug 12 16:51:03.289540049 (95h24m57.7206202s) [upload=99416]: Upload finished. err: <nil> Duration: 5.983852063s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.X.............................X.X.........X..........X.....] duration: 5.964108424s FAIL (2025-08-12 17:12:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after b0982c89d4a14d25bd833df87489bff9fd295b7477379b6d824f265e357cc0f2 != before 197684ff454ac1a3a74c23a2c042188f74f051027843e0f6c54d853073e620af ======LOGS====== Aug 12 17:09:40.321853865 (95h43m34.752933996s) [upload=99722]: Starting upload Aug 12 17:09:40.321893551 (95h43m34.752973682s) [upload=99722]: Done starting upload Aug 12 17:09:40.321966250 (95h43m34.753046381s) [upload=99722]: Uploading segments... Aug 12 17:09:40.322074217 (95h43m34.753154348s) [upload=99722]: Got next segment. Inline: false Aug 12 17:09:40.322110977 (95h43m34.753191108s) [upload=99722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 17:09:40.322117018 (95h43m34.753197149s) [upload=99722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 17:09:40.774677309 (95h43m35.20575744s) [seg_pos={0 0}, upload=99722]: Begin upload segment... Aug 12 17:09:40.774728086 (95h43m35.205808217s) [seg_pos={0 0}, upload=99722]: Done begin upload segment. Aug 12 17:09:40.774751501 (95h43m35.205831622s) [noise=true , address=104.36.82.14:4233 , piece_id=G57MOVDGURNTF3GII4PG6INCDVLEFHQG4ZKECN4RDFQ3ZOFJJMPQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.774889235 (95h43m35.205969436s) [noise=true , address=104.36.82.34:4562 , piece_id=XSQXN4UZ4XALRCL2HC4SH63Q7VX6OFS63PILMNVLW6YUJ3SDZTQQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775115477 (95h43m35.206195608s) [noise=true , address=104.36.82.60:4358 , piece_id=6WK4MJI4MT6BWYXUQDPWKKEG6HVRHHN66IF27B3RCVA2K3IEEGEQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775394030 (95h43m35.206474161s) [noise=true , address=104.36.82.61:4388 , piece_id=MJL5EIV4QAGB57TYHAVPJWMRNJ2NV2XI3PDU2KG2NRUFOU5YX66Q, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775612628 (95h43m35.20669276s) [noise=true , address=104.36.82.20:4446 , piece_id=STM6GTU5C7KSEZKNEQDBBKG3KUESFA66XI4M7C5F2NDIOIMXRURA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775684887 (95h43m35.206765018s) [noise=true , address=45.91.116.5:4323 , piece_id=7OWV6OPJPRGM5MPZ25ULPBRADADRT5QMFXRJYNTDJNHJUC566WJA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775929696 (95h43m35.207009817s) [noise=true , address=79.127.201.215:4349 , piece_id=DWHXJZ6JYHNFZ7SYWLDBXKPPFEJ6NYASMN5CVBAVDGVYQURQ3PXA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776002796 (95h43m35.207082917s) [noise=true , address=104.36.82.7:4919 , piece_id=ARXPHRRZZ2OMTFTMUSHZFZ7ONGOZU3JEVUAAVZVFHOCUFBOJE6AA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776223618 (95h43m35.207303749s) [noise=true , address=104.36.82.46:4500 , piece_id=CRSOKFHEYK2G4NOSRZNJ3LYPJ5TLYAQHCRY2K36DTKKYYQQHRLDA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776298280 (95h43m35.207378401s) [noise=true , address=77.90.31.12:4623 , piece_id=HATORCXVWJQF5K2VFTDI52NIHMMWTDZDIMSFNGOSVFIIJARUJJWQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776500628 (95h43m35.207580759s) [noise=true , address=104.36.82.36:4624 , piece_id=EHDUBB4CI2FSOOV5WRWHLXRNSZOLS6NJVLZOWGNFNUKDASABC56A, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776585781 (95h43m35.207665912s) [noise=true , address=104.36.82.12:4159 , piece_id=U5JLWCP2MNCMXDEVPPESXRDJYKAYZQXIP2DQ3VDHUUNVSOPDWUJA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776747651 (95h43m35.207827782s) [noise=true , address=104.36.82.8:4995 , piece_id=3SD4HYUCN3Y7YAQP6IFRZKN2A2KD2YP2BDK3FDKRVEG3RTI6KQ4A, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776809178 (95h43m35.207889309s) [noise=true , address=104.36.82.58:4282 , piece_id=FKV7JWSXMZOBTOSEJIG4THDZVD4CTQNWESMC3AZG7UAEE7SQWUZA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.777003861 (95h43m35.208083992s) [noise=true , address=77.90.31.19:4865 , piece_id=KEEXRKILFITPIXVHQIJ5WEA3N5JWZR4SOYX2YKQXX5RIHAGPEAIA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.777030371 (95h43m35.208110503s) [noise=true , address=104.36.82.47:4524 , piece_id=VPOIP7WFL27O37X5JK4YOWMRC3ZROKBJYGLGJE2YPZNL7L7FXPKQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.777074015 (95h43m35.208154146s) [noise=true , address=104.36.82.17:4359 , piece_id=ZE6ZCIFIGRIRPCSTOXGYKICSMQ4MRW5EOVNNVD4X5WW2YOM6IYCA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.777244772 (95h43m35.208324903s) [noise=true , address=104.36.82.23:4475 , piece_id=2LS7X3RFXUFYRPCUNSJCANN2I6QYM2EEYWXEO7IN7GHKEYX45CWQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:42.295179902 (95h43m36.726260033s) [upload=99722]: Next returned nil segment Aug 12 17:09:42.295194640 (95h43m36.726274761s) [upload=99722]: Waiting for error group managing segments... Aug 12 17:09:42.729495544 (95h43m37.160575665s) [noise=true , address=45.91.116.5:4323 , piece_id=7OWV6OPJPRGM5MPZ25ULPBRADADRT5QMFXRJYNTDJNHJUC566WJA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.732790118 (95h43m37.163870289s) [noise=true , address=104.36.82.34:4562 , piece_id=XSQXN4UZ4XALRCL2HC4SH63Q7VX6OFS63PILMNVLW6YUJ3SDZTQQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.747195941 (95h43m37.178276072s) [noise=true , address=104.36.82.58:4282 , piece_id=FKV7JWSXMZOBTOSEJIG4THDZVD4CTQNWESMC3AZG7UAEE7SQWUZA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.754626518 (95h43m37.18570665s) [noise=true , address=104.36.82.7:4919 , piece_id=ARXPHRRZZ2OMTFTMUSHZFZ7ONGOZU3JEVUAAVZVFHOCUFBOJE6AA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.791808389 (95h43m37.22288852s) [noise=true , address=104.36.82.47:4524 , piece_id=VPOIP7WFL27O37X5JK4YOWMRC3ZROKBJYGLGJE2YPZNL7L7FXPKQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.803351946 (95h43m37.234432067s) [noise=true , address=104.36.82.20:4446 , piece_id=STM6GTU5C7KSEZKNEQDBBKG3KUESFA66XI4M7C5F2NDIOIMXRURA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.846122721 (95h43m37.277202852s) [noise=true , address=104.36.82.12:4159 , piece_id=U5JLWCP2MNCMXDEVPPESXRDJYKAYZQXIP2DQ3VDHUUNVSOPDWUJA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.856142241 (95h43m37.287222373s) [noise=true , address=104.36.82.14:4233 , piece_id=G57MOVDGURNTF3GII4PG6INCDVLEFHQG4ZKECN4RDFQ3ZOFJJMPQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.860973705 (95h43m37.292053836s) [noise=true , address=104.36.82.8:4995 , piece_id=3SD4HYUCN3Y7YAQP6IFRZKN2A2KD2YP2BDK3FDKRVEG3RTI6KQ4A, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.880185945 (95h43m37.311266076s) [noise=true , address=104.36.82.23:4475 , piece_id=2LS7X3RFXUFYRPCUNSJCANN2I6QYM2EEYWXEO7IN7GHKEYX45CWQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.881755528 (95h43m37.312835659s) [noise=true , address=104.36.82.17:4359 , piece_id=ZE6ZCIFIGRIRPCSTOXGYKICSMQ4MRW5EOVNNVD4X5WW2YOM6IYCA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.889261359 (95h43m37.32034149s) [noise=true , address=104.36.82.46:4500 , piece_id=CRSOKFHEYK2G4NOSRZNJ3LYPJ5TLYAQHCRY2K36DTKKYYQQHRLDA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.943274955 (95h43m37.374355116s) [noise=true , address=104.36.82.61:4388 , piece_id=MJL5EIV4QAGB57TYHAVPJWMRNJ2NV2XI3PDU2KG2NRUFOU5YX66Q, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.952686322 (95h43m37.383766453s) [noise=true , address=104.36.82.36:4624 , piece_id=EHDUBB4CI2FSOOV5WRWHLXRNSZOLS6NJVLZOWGNFNUKDASABC56A, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.952709246 (95h43m37.383789377s) [seg_pos={0 0}, upload=99722]: Segment reached optimal threshold of 14 pieces. Aug 12 17:09:42.952895372 (95h43m37.383975504s) [noise=true , address=77.90.31.19:4865 , piece_id=KEEXRKILFITPIXVHQIJ5WEA3N5JWZR4SOYX2YKQXX5RIHAGPEAIA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bguBghQonV5sTHaWteyUEnpp5KNzBKbvBpbvrUNFDCveHN77X): protocol: expected piece hash; context canceled; EOF Aug 12 17:09:42.952962120 (95h43m37.384042251s) [noise=true , address=79.127.201.215:4349 , piece_id=DWHXJZ6JYHNFZ7SYWLDBXKPPFEJ6NYASMN5CVBAVDGVYQURQ3PXA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y1KYQJsvbNxfySU7mXySUmJtEDVAE7fst1XVRHU5N8Txmjn5T): protocol: expected piece hash; context canceled; EOF Aug 12 17:09:42.952984222 (95h43m37.384064353s) [noise=true , address=77.90.31.12:4623 , piece_id=HATORCXVWJQF5K2VFTDI52NIHMMWTDZDIMSFNGOSVFIIJARUJJWQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: expected piece hash; context canceled; EOF Aug 12 17:09:42.953037214 (95h43m37.384117345s) [noise=true , address=104.36.82.60:4358 , piece_id=6WK4MJI4MT6BWYXUQDPWKKEG6HVRHHN66IF27B3RCVA2K3IEEGEQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dwCNEcA9o51ZN3UaN3Frxdawwf8DXBj86iQMywjW8XP4ESmw1): protocol: write tcp 188.166.225.170:43124->104.36.82.60:4358: use of closed network connection; write tcp 188.166.225.170:43124->104.36.82.60:4358: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43124->104.36.82.60:4358: use of closed network connection Aug 12 17:09:42.953048665 (95h43m37.384128786s) [seg_pos={0 0}, upload=99722]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 12 17:09:42.953101797 (95h43m37.384181928s) [upload=99722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 17:09:42.953105504 (95h43m37.384185635s) [upload=99722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 17:09:43.248448404 (95h43m37.679528535s) [upload=99722]: Done uploading segments... Aug 12 17:09:43.248466649 (95h43m37.67954677s) [upload=99722]: Upload finished. err: <nil> Duration: 5.964136587s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [......X..........X..........................................] duration: 1.847434377s FAIL (2025-08-12 17:12:30 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 12 16:39:09.008146755 (95h13m3.439226887s) [upload=99194]: Starting upload Aug 12 16:39:09.008201260 (95h13m3.439281391s) [upload=99194]: Done starting upload Aug 12 16:39:09.008258329 (95h13m3.43933846s) [upload=99194]: Uploading segments... Aug 12 16:39:09.008290200 (95h13m3.439370331s) [upload=99194]: Got next segment. Inline: true Aug 12 16:39:09.008295661 (95h13m3.439375782s) [upload=99194]: Waiting for error group managing segments... Aug 12 16:39:09.008328123 (95h13m3.439408254s) [upload=99194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:39:09.008334585 (95h13m3.439414716s) [upload=99194]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 16:39:09.008337160 (95h13m3.439417281s) [upload=99194]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:39:09.337170261 (95h13m3.768250392s) [upload=99194]: Done uploading segments... Aug 12 16:39:09.337190520 (95h13m3.768270651s) [upload=99194]: Upload finished. err: <nil> Duration:* 1.84746743s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.025813449s FAIL (2025-08-12 17:12:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 12 16:51:00.342944596 (95h24m54.774024727s) [upload=99416]: Starting upload Aug 12 16:51:00.343003058 (95h24m54.774083189s) [upload=99416]: Done starting upload Aug 12 16:51:00.343093682 (95h24m54.774173803s) [upload=99416]: Uploading segments... Aug 12 16:51:00.343214823 (95h24m54.774294954s) [upload=99416]: Got next segment. Inline: false Aug 12 16:51:00.343269027 (95h24m54.774349158s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:51:00.343273756 (95h24m54.774353887s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 16:51:00.585349151 (95h24m55.016429282s) [seg_pos={0 0}, upload=99416]: Begin upload segment... Aug 12 16:51:00.585396822 (95h24m55.016476973s) [seg_pos={0 0}, upload=99416]: Done begin upload segment. Aug 12 16:51:00.585410067 (95h24m55.016490208s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585507734 (95h24m55.016587895s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585886159 (95h24m55.0169663s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586089607 (95h24m55.017169738s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586541652 (95h24m55.017621803s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587004688 (95h24m55.018084819s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587619234 (95h24m55.018699385s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587983842 (95h24m55.019063983s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588303153 (95h24m55.019383284s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588688090 (95h24m55.019768221s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.589902393 (95h24m55.020982534s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.590674450 (95h24m55.021754601s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.591296961 (95h24m55.022377092s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.592189969 (95h24m55.02327011s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.593156619 (95h24m55.02423678s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.594056290 (95h24m55.025136421s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:02.362133336 (95h24m56.793213467s) [upload=99416]: Next returned nil segment Aug 12 16:51:02.362159547 (95h24m56.793239678s) [upload=99416]: Waiting for error group managing segments... Aug 12 16:51:02.650293798 (95h24m57.081373929s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.663772758 (95h24m57.094852919s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.696344749 (95h24m57.12742488s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.709104884 (95h24m57.140185025s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.722233484 (95h24m57.153313625s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.727048907 (95h24m57.158129038s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.741870376 (95h24m57.172950507s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.772732365 (95h24m57.203812507s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.805755481 (95h24m57.236835612s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.878280266 (95h24m57.309360397s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.913734542 (95h24m57.344814673s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.972369120 (95h24m57.403449251s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983815812 (95h24m57.414895943s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983842733 (95h24m57.414922864s) [seg_pos={0 0}, upload=99416]: Segment reached optimal threshold of 13 pieces. Aug 12 16:51:02.983942084 (95h24m57.415022215s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.983999583 (95h24m57.415079715s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984087211 (95h24m57.415167342s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984124363 (95h24m57.415204494s) [seg_pos={0 0}, upload=99416]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 12 16:51:02.984229683 (95h24m57.415309814s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 16:51:02.984234323 (95h24m57.415314454s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:51:03.289516404 (95h24m57.720596575s) [upload=99416]: Done uploading segments... Aug 12 16:51:03.289540049 (95h24m57.7206202s) [upload=99416]: Upload finished. err: <nil> Duration: 6.025836915s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..X.............................X.X.........X..........X....] duration: 6.003441962s FAIL (2025-08-12 17:12:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after b0982c89d4a14d25bd833df87489bff9fd295b7477379b6d824f265e357cc0f2 != before 197684ff454ac1a3a74c23a2c042188f74f051027843e0f6c54d853073e620af ======LOGS====== Aug 12 17:09:40.321853865 (95h43m34.752933996s) [upload=99722]: Starting upload Aug 12 17:09:40.321893551 (95h43m34.752973682s) [upload=99722]: Done starting upload Aug 12 17:09:40.321966250 (95h43m34.753046381s) [upload=99722]: Uploading segments... Aug 12 17:09:40.322074217 (95h43m34.753154348s) [upload=99722]: Got next segment. Inline: false Aug 12 17:09:40.322110977 (95h43m34.753191108s) [upload=99722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 17:09:40.322117018 (95h43m34.753197149s) [upload=99722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 17:09:40.774677309 (95h43m35.20575744s) [seg_pos={0 0}, upload=99722]: Begin upload segment... Aug 12 17:09:40.774728086 (95h43m35.205808217s) [seg_pos={0 0}, upload=99722]: Done begin upload segment. Aug 12 17:09:40.774751501 (95h43m35.205831622s) [noise=true , address=104.36.82.14:4233 , piece_id=G57MOVDGURNTF3GII4PG6INCDVLEFHQG4ZKECN4RDFQ3ZOFJJMPQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.774889235 (95h43m35.205969436s) [noise=true , address=104.36.82.34:4562 , piece_id=XSQXN4UZ4XALRCL2HC4SH63Q7VX6OFS63PILMNVLW6YUJ3SDZTQQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775115477 (95h43m35.206195608s) [noise=true , address=104.36.82.60:4358 , piece_id=6WK4MJI4MT6BWYXUQDPWKKEG6HVRHHN66IF27B3RCVA2K3IEEGEQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775394030 (95h43m35.206474161s) [noise=true , address=104.36.82.61:4388 , piece_id=MJL5EIV4QAGB57TYHAVPJWMRNJ2NV2XI3PDU2KG2NRUFOU5YX66Q, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775612628 (95h43m35.20669276s) [noise=true , address=104.36.82.20:4446 , piece_id=STM6GTU5C7KSEZKNEQDBBKG3KUESFA66XI4M7C5F2NDIOIMXRURA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775684887 (95h43m35.206765018s) [noise=true , address=45.91.116.5:4323 , piece_id=7OWV6OPJPRGM5MPZ25ULPBRADADRT5QMFXRJYNTDJNHJUC566WJA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775929696 (95h43m35.207009817s) [noise=true , address=79.127.201.215:4349 , piece_id=DWHXJZ6JYHNFZ7SYWLDBXKPPFEJ6NYASMN5CVBAVDGVYQURQ3PXA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776002796 (95h43m35.207082917s) [noise=true , address=104.36.82.7:4919 , piece_id=ARXPHRRZZ2OMTFTMUSHZFZ7ONGOZU3JEVUAAVZVFHOCUFBOJE6AA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776223618 (95h43m35.207303749s) [noise=true , address=104.36.82.46:4500 , piece_id=CRSOKFHEYK2G4NOSRZNJ3LYPJ5TLYAQHCRY2K36DTKKYYQQHRLDA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776298280 (95h43m35.207378401s) [noise=true , address=77.90.31.12:4623 , piece_id=HATORCXVWJQF5K2VFTDI52NIHMMWTDZDIMSFNGOSVFIIJARUJJWQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776500628 (95h43m35.207580759s) [noise=true , address=104.36.82.36:4624 , piece_id=EHDUBB4CI2FSOOV5WRWHLXRNSZOLS6NJVLZOWGNFNUKDASABC56A, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776585781 (95h43m35.207665912s) [noise=true , address=104.36.82.12:4159 , piece_id=U5JLWCP2MNCMXDEVPPESXRDJYKAYZQXIP2DQ3VDHUUNVSOPDWUJA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776747651 (95h43m35.207827782s) [noise=true , address=104.36.82.8:4995 , piece_id=3SD4HYUCN3Y7YAQP6IFRZKN2A2KD2YP2BDK3FDKRVEG3RTI6KQ4A, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776809178 (95h43m35.207889309s) [noise=true , address=104.36.82.58:4282 , piece_id=FKV7JWSXMZOBTOSEJIG4THDZVD4CTQNWESMC3AZG7UAEE7SQWUZA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.777003861 (95h43m35.208083992s) [noise=true , address=77.90.31.19:4865 , piece_id=KEEXRKILFITPIXVHQIJ5WEA3N5JWZR4SOYX2YKQXX5RIHAGPEAIA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.777030371 (95h43m35.208110503s) [noise=true , address=104.36.82.47:4524 , piece_id=VPOIP7WFL27O37X5JK4YOWMRC3ZROKBJYGLGJE2YPZNL7L7FXPKQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.777074015 (95h43m35.208154146s) [noise=true , address=104.36.82.17:4359 , piece_id=ZE6ZCIFIGRIRPCSTOXGYKICSMQ4MRW5EOVNNVD4X5WW2YOM6IYCA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.777244772 (95h43m35.208324903s) [noise=true , address=104.36.82.23:4475 , piece_id=2LS7X3RFXUFYRPCUNSJCANN2I6QYM2EEYWXEO7IN7GHKEYX45CWQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:42.295179902 (95h43m36.726260033s) [upload=99722]: Next returned nil segment Aug 12 17:09:42.295194640 (95h43m36.726274761s) [upload=99722]: Waiting for error group managing segments... Aug 12 17:09:42.729495544 (95h43m37.160575665s) [noise=true , address=45.91.116.5:4323 , piece_id=7OWV6OPJPRGM5MPZ25ULPBRADADRT5QMFXRJYNTDJNHJUC566WJA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.732790118 (95h43m37.163870289s) [noise=true , address=104.36.82.34:4562 , piece_id=XSQXN4UZ4XALRCL2HC4SH63Q7VX6OFS63PILMNVLW6YUJ3SDZTQQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.747195941 (95h43m37.178276072s) [noise=true , address=104.36.82.58:4282 , piece_id=FKV7JWSXMZOBTOSEJIG4THDZVD4CTQNWESMC3AZG7UAEE7SQWUZA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.754626518 (95h43m37.18570665s) [noise=true , address=104.36.82.7:4919 , piece_id=ARXPHRRZZ2OMTFTMUSHZFZ7ONGOZU3JEVUAAVZVFHOCUFBOJE6AA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.791808389 (95h43m37.22288852s) [noise=true , address=104.36.82.47:4524 , piece_id=VPOIP7WFL27O37X5JK4YOWMRC3ZROKBJYGLGJE2YPZNL7L7FXPKQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.803351946 (95h43m37.234432067s) [noise=true , address=104.36.82.20:4446 , piece_id=STM6GTU5C7KSEZKNEQDBBKG3KUESFA66XI4M7C5F2NDIOIMXRURA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.846122721 (95h43m37.277202852s) [noise=true , address=104.36.82.12:4159 , piece_id=U5JLWCP2MNCMXDEVPPESXRDJYKAYZQXIP2DQ3VDHUUNVSOPDWUJA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.856142241 (95h43m37.287222373s) [noise=true , address=104.36.82.14:4233 , piece_id=G57MOVDGURNTF3GII4PG6INCDVLEFHQG4ZKECN4RDFQ3ZOFJJMPQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.860973705 (95h43m37.292053836s) [noise=true , address=104.36.82.8:4995 , piece_id=3SD4HYUCN3Y7YAQP6IFRZKN2A2KD2YP2BDK3FDKRVEG3RTI6KQ4A, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.880185945 (95h43m37.311266076s) [noise=true , address=104.36.82.23:4475 , piece_id=2LS7X3RFXUFYRPCUNSJCANN2I6QYM2EEYWXEO7IN7GHKEYX45CWQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.881755528 (95h43m37.312835659s) [noise=true , address=104.36.82.17:4359 , piece_id=ZE6ZCIFIGRIRPCSTOXGYKICSMQ4MRW5EOVNNVD4X5WW2YOM6IYCA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.889261359 (95h43m37.32034149s) [noise=true , address=104.36.82.46:4500 , piece_id=CRSOKFHEYK2G4NOSRZNJ3LYPJ5TLYAQHCRY2K36DTKKYYQQHRLDA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.943274955 (95h43m37.374355116s) [noise=true , address=104.36.82.61:4388 , piece_id=MJL5EIV4QAGB57TYHAVPJWMRNJ2NV2XI3PDU2KG2NRUFOU5YX66Q, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.952686322 (95h43m37.383766453s) [noise=true , address=104.36.82.36:4624 , piece_id=EHDUBB4CI2FSOOV5WRWHLXRNSZOLS6NJVLZOWGNFNUKDASABC56A, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.952709246 (95h43m37.383789377s) [seg_pos={0 0}, upload=99722]: Segment reached optimal threshold of 14 pieces. Aug 12 17:09:42.952895372 (95h43m37.383975504s) [noise=true , address=77.90.31.19:4865 , piece_id=KEEXRKILFITPIXVHQIJ5WEA3N5JWZR4SOYX2YKQXX5RIHAGPEAIA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bguBghQonV5sTHaWteyUEnpp5KNzBKbvBpbvrUNFDCveHN77X): protocol: expected piece hash; context canceled; EOF Aug 12 17:09:42.952962120 (95h43m37.384042251s) [noise=true , address=79.127.201.215:4349 , piece_id=DWHXJZ6JYHNFZ7SYWLDBXKPPFEJ6NYASMN5CVBAVDGVYQURQ3PXA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y1KYQJsvbNxfySU7mXySUmJtEDVAE7fst1XVRHU5N8Txmjn5T): protocol: expected piece hash; context canceled; EOF Aug 12 17:09:42.952984222 (95h43m37.384064353s) [noise=true , address=77.90.31.12:4623 , piece_id=HATORCXVWJQF5K2VFTDI52NIHMMWTDZDIMSFNGOSVFIIJARUJJWQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: expected piece hash; context canceled; EOF Aug 12 17:09:42.953037214 (95h43m37.384117345s) [noise=true , address=104.36.82.60:4358 , piece_id=6WK4MJI4MT6BWYXUQDPWKKEG6HVRHHN66IF27B3RCVA2K3IEEGEQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dwCNEcA9o51ZN3UaN3Frxdawwf8DXBj86iQMywjW8XP4ESmw1): protocol: write tcp 188.166.225.170:43124->104.36.82.60:4358: use of closed network connection; write tcp 188.166.225.170:43124->104.36.82.60:4358: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43124->104.36.82.60:4358: use of closed network connection Aug 12 17:09:42.953048665 (95h43m37.384128786s) [seg_pos={0 0}, upload=99722]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 12 17:09:42.953101797 (95h43m37.384181928s) [upload=99722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 17:09:42.953105504 (95h43m37.384185635s) [upload=99722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 17:09:43.248448404 (95h43m37.679528535s) [upload=99722]: Done uploading segments... Aug 12 17:09:43.248466649 (95h43m37.67954677s) [upload=99722]: Upload finished. err: <nil> Duration: 6.003464234s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...X.............................X.X.........X..........X...] duration: 5.978025245s FAIL (2025-08-12 17:11:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after b0982c89d4a14d25bd833df87489bff9fd295b7477379b6d824f265e357cc0f2 != before 197684ff454ac1a3a74c23a2c042188f74f051027843e0f6c54d853073e620af ======LOGS====== Aug 12 17:09:40.321853865 (95h43m34.752933996s) [upload=99722]: Starting upload Aug 12 17:09:40.321893551 (95h43m34.752973682s) [upload=99722]: Done starting upload Aug 12 17:09:40.321966250 (95h43m34.753046381s) [upload=99722]: Uploading segments... Aug 12 17:09:40.322074217 (95h43m34.753154348s) [upload=99722]: Got next segment. Inline: false Aug 12 17:09:40.322110977 (95h43m34.753191108s) [upload=99722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 17:09:40.322117018 (95h43m34.753197149s) [upload=99722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 17:09:40.774677309 (95h43m35.20575744s) [seg_pos={0 0}, upload=99722]: Begin upload segment... Aug 12 17:09:40.774728086 (95h43m35.205808217s) [seg_pos={0 0}, upload=99722]: Done begin upload segment. Aug 12 17:09:40.774751501 (95h43m35.205831622s) [noise=true , address=104.36.82.14:4233 , piece_id=G57MOVDGURNTF3GII4PG6INCDVLEFHQG4ZKECN4RDFQ3ZOFJJMPQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.774889235 (95h43m35.205969436s) [noise=true , address=104.36.82.34:4562 , piece_id=XSQXN4UZ4XALRCL2HC4SH63Q7VX6OFS63PILMNVLW6YUJ3SDZTQQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775115477 (95h43m35.206195608s) [noise=true , address=104.36.82.60:4358 , piece_id=6WK4MJI4MT6BWYXUQDPWKKEG6HVRHHN66IF27B3RCVA2K3IEEGEQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775394030 (95h43m35.206474161s) [noise=true , address=104.36.82.61:4388 , piece_id=MJL5EIV4QAGB57TYHAVPJWMRNJ2NV2XI3PDU2KG2NRUFOU5YX66Q, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775612628 (95h43m35.20669276s) [noise=true , address=104.36.82.20:4446 , piece_id=STM6GTU5C7KSEZKNEQDBBKG3KUESFA66XI4M7C5F2NDIOIMXRURA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775684887 (95h43m35.206765018s) [noise=true , address=45.91.116.5:4323 , piece_id=7OWV6OPJPRGM5MPZ25ULPBRADADRT5QMFXRJYNTDJNHJUC566WJA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775929696 (95h43m35.207009817s) [noise=true , address=79.127.201.215:4349 , piece_id=DWHXJZ6JYHNFZ7SYWLDBXKPPFEJ6NYASMN5CVBAVDGVYQURQ3PXA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776002796 (95h43m35.207082917s) [noise=true , address=104.36.82.7:4919 , piece_id=ARXPHRRZZ2OMTFTMUSHZFZ7ONGOZU3JEVUAAVZVFHOCUFBOJE6AA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776223618 (95h43m35.207303749s) [noise=true , address=104.36.82.46:4500 , piece_id=CRSOKFHEYK2G4NOSRZNJ3LYPJ5TLYAQHCRY2K36DTKKYYQQHRLDA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776298280 (95h43m35.207378401s) [noise=true , address=77.90.31.12:4623 , piece_id=HATORCXVWJQF5K2VFTDI52NIHMMWTDZDIMSFNGOSVFIIJARUJJWQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776500628 (95h43m35.207580759s) [noise=true , address=104.36.82.36:4624 , piece_id=EHDUBB4CI2FSOOV5WRWHLXRNSZOLS6NJVLZOWGNFNUKDASABC56A, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776585781 (95h43m35.207665912s) [noise=true , address=104.36.82.12:4159 , piece_id=U5JLWCP2MNCMXDEVPPESXRDJYKAYZQXIP2DQ3VDHUUNVSOPDWUJA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776747651 (95h43m35.207827782s) [noise=true , address=104.36.82.8:4995 , piece_id=3SD4HYUCN3Y7YAQP6IFRZKN2A2KD2YP2BDK3FDKRVEG3RTI6KQ4A, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776809178 (95h43m35.207889309s) [noise=true , address=104.36.82.58:4282 , piece_id=FKV7JWSXMZOBTOSEJIG4THDZVD4CTQNWESMC3AZG7UAEE7SQWUZA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.777003861 (95h43m35.208083992s) [noise=true , address=77.90.31.19:4865 , piece_id=KEEXRKILFITPIXVHQIJ5WEA3N5JWZR4SOYX2YKQXX5RIHAGPEAIA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.777030371 (95h43m35.208110503s) [noise=true , address=104.36.82.47:4524 , piece_id=VPOIP7WFL27O37X5JK4YOWMRC3ZROKBJYGLGJE2YPZNL7L7FXPKQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.777074015 (95h43m35.208154146s) [noise=true , address=104.36.82.17:4359 , piece_id=ZE6ZCIFIGRIRPCSTOXGYKICSMQ4MRW5EOVNNVD4X5WW2YOM6IYCA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.777244772 (95h43m35.208324903s) [noise=true , address=104.36.82.23:4475 , piece_id=2LS7X3RFXUFYRPCUNSJCANN2I6QYM2EEYWXEO7IN7GHKEYX45CWQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:42.295179902 (95h43m36.726260033s) [upload=99722]: Next returned nil segment Aug 12 17:09:42.295194640 (95h43m36.726274761s) [upload=99722]: Waiting for error group managing segments... Aug 12 17:09:42.729495544 (95h43m37.160575665s) [noise=true , address=45.91.116.5:4323 , piece_id=7OWV6OPJPRGM5MPZ25ULPBRADADRT5QMFXRJYNTDJNHJUC566WJA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.732790118 (95h43m37.163870289s) [noise=true , address=104.36.82.34:4562 , piece_id=XSQXN4UZ4XALRCL2HC4SH63Q7VX6OFS63PILMNVLW6YUJ3SDZTQQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.747195941 (95h43m37.178276072s) [noise=true , address=104.36.82.58:4282 , piece_id=FKV7JWSXMZOBTOSEJIG4THDZVD4CTQNWESMC3AZG7UAEE7SQWUZA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.754626518 (95h43m37.18570665s) [noise=true , address=104.36.82.7:4919 , piece_id=ARXPHRRZZ2OMTFTMUSHZFZ7ONGOZU3JEVUAAVZVFHOCUFBOJE6AA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.791808389 (95h43m37.22288852s) [noise=true , address=104.36.82.47:4524 , piece_id=VPOIP7WFL27O37X5JK4YOWMRC3ZROKBJYGLGJE2YPZNL7L7FXPKQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.803351946 (95h43m37.234432067s) [noise=true , address=104.36.82.20:4446 , piece_id=STM6GTU5C7KSEZKNEQDBBKG3KUESFA66XI4M7C5F2NDIOIMXRURA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.846122721 (95h43m37.277202852s) [noise=true , address=104.36.82.12:4159 , piece_id=U5JLWCP2MNCMXDEVPPESXRDJYKAYZQXIP2DQ3VDHUUNVSOPDWUJA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.856142241 (95h43m37.287222373s) [noise=true , address=104.36.82.14:4233 , piece_id=G57MOVDGURNTF3GII4PG6INCDVLEFHQG4ZKECN4RDFQ3ZOFJJMPQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.860973705 (95h43m37.292053836s) [noise=true , address=104.36.82.8:4995 , piece_id=3SD4HYUCN3Y7YAQP6IFRZKN2A2KD2YP2BDK3FDKRVEG3RTI6KQ4A, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.880185945 (95h43m37.311266076s) [noise=true , address=104.36.82.23:4475 , piece_id=2LS7X3RFXUFYRPCUNSJCANN2I6QYM2EEYWXEO7IN7GHKEYX45CWQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.881755528 (95h43m37.312835659s) [noise=true , address=104.36.82.17:4359 , piece_id=ZE6ZCIFIGRIRPCSTOXGYKICSMQ4MRW5EOVNNVD4X5WW2YOM6IYCA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.889261359 (95h43m37.32034149s) [noise=true , address=104.36.82.46:4500 , piece_id=CRSOKFHEYK2G4NOSRZNJ3LYPJ5TLYAQHCRY2K36DTKKYYQQHRLDA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.943274955 (95h43m37.374355116s) [noise=true , address=104.36.82.61:4388 , piece_id=MJL5EIV4QAGB57TYHAVPJWMRNJ2NV2XI3PDU2KG2NRUFOU5YX66Q, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.952686322 (95h43m37.383766453s) [noise=true , address=104.36.82.36:4624 , piece_id=EHDUBB4CI2FSOOV5WRWHLXRNSZOLS6NJVLZOWGNFNUKDASABC56A, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.952709246 (95h43m37.383789377s) [seg_pos={0 0}, upload=99722]: Segment reached optimal threshold of 14 pieces. Aug 12 17:09:42.952895372 (95h43m37.383975504s) [noise=true , address=77.90.31.19:4865 , piece_id=KEEXRKILFITPIXVHQIJ5WEA3N5JWZR4SOYX2YKQXX5RIHAGPEAIA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bguBghQonV5sTHaWteyUEnpp5KNzBKbvBpbvrUNFDCveHN77X): protocol: expected piece hash; context canceled; EOF Aug 12 17:09:42.952962120 (95h43m37.384042251s) [noise=true , address=79.127.201.215:4349 , piece_id=DWHXJZ6JYHNFZ7SYWLDBXKPPFEJ6NYASMN5CVBAVDGVYQURQ3PXA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y1KYQJsvbNxfySU7mXySUmJtEDVAE7fst1XVRHU5N8Txmjn5T): protocol: expected piece hash; context canceled; EOF Aug 12 17:09:42.952984222 (95h43m37.384064353s) [noise=true , address=77.90.31.12:4623 , piece_id=HATORCXVWJQF5K2VFTDI52NIHMMWTDZDIMSFNGOSVFIIJARUJJWQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: expected piece hash; context canceled; EOF Aug 12 17:09:42.953037214 (95h43m37.384117345s) [noise=true , address=104.36.82.60:4358 , piece_id=6WK4MJI4MT6BWYXUQDPWKKEG6HVRHHN66IF27B3RCVA2K3IEEGEQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dwCNEcA9o51ZN3UaN3Frxdawwf8DXBj86iQMywjW8XP4ESmw1): protocol: write tcp 188.166.225.170:43124->104.36.82.60:4358: use of closed network connection; write tcp 188.166.225.170:43124->104.36.82.60:4358: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43124->104.36.82.60:4358: use of closed network connection Aug 12 17:09:42.953048665 (95h43m37.384128786s) [seg_pos={0 0}, upload=99722]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 12 17:09:42.953101797 (95h43m37.384181928s) [upload=99722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 17:09:42.953105504 (95h43m37.384185635s) [upload=99722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 17:09:43.248448404 (95h43m37.679528535s) [upload=99722]: Done uploading segments... Aug 12 17:09:43.248466649 (95h43m37.67954677s) [upload=99722]: Upload finished. err: <nil> Duration: 5.978058709s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.883279698s FAIL (2025-08-12 17:11:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 12 16:51:00.342944596 (95h24m54.774024727s) [upload=99416]: Starting upload Aug 12 16:51:00.343003058 (95h24m54.774083189s) [upload=99416]: Done starting upload Aug 12 16:51:00.343093682 (95h24m54.774173803s) [upload=99416]: Uploading segments... Aug 12 16:51:00.343214823 (95h24m54.774294954s) [upload=99416]: Got next segment. Inline: false Aug 12 16:51:00.343269027 (95h24m54.774349158s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:51:00.343273756 (95h24m54.774353887s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 16:51:00.585349151 (95h24m55.016429282s) [seg_pos={0 0}, upload=99416]: Begin upload segment... Aug 12 16:51:00.585396822 (95h24m55.016476973s) [seg_pos={0 0}, upload=99416]: Done begin upload segment. Aug 12 16:51:00.585410067 (95h24m55.016490208s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585507734 (95h24m55.016587895s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585886159 (95h24m55.0169663s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586089607 (95h24m55.017169738s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586541652 (95h24m55.017621803s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587004688 (95h24m55.018084819s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587619234 (95h24m55.018699385s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587983842 (95h24m55.019063983s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588303153 (95h24m55.019383284s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588688090 (95h24m55.019768221s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.589902393 (95h24m55.020982534s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.590674450 (95h24m55.021754601s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.591296961 (95h24m55.022377092s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.592189969 (95h24m55.02327011s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.593156619 (95h24m55.02423678s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.594056290 (95h24m55.025136421s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:02.362133336 (95h24m56.793213467s) [upload=99416]: Next returned nil segment Aug 12 16:51:02.362159547 (95h24m56.793239678s) [upload=99416]: Waiting for error group managing segments... Aug 12 16:51:02.650293798 (95h24m57.081373929s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.663772758 (95h24m57.094852919s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.696344749 (95h24m57.12742488s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.709104884 (95h24m57.140185025s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.722233484 (95h24m57.153313625s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.727048907 (95h24m57.158129038s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.741870376 (95h24m57.172950507s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.772732365 (95h24m57.203812507s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.805755481 (95h24m57.236835612s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.878280266 (95h24m57.309360397s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.913734542 (95h24m57.344814673s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.972369120 (95h24m57.403449251s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983815812 (95h24m57.414895943s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983842733 (95h24m57.414922864s) [seg_pos={0 0}, upload=99416]: Segment reached optimal threshold of 13 pieces. Aug 12 16:51:02.983942084 (95h24m57.415022215s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.983999583 (95h24m57.415079715s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984087211 (95h24m57.415167342s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984124363 (95h24m57.415204494s) [seg_pos={0 0}, upload=99416]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 12 16:51:02.984229683 (95h24m57.415309814s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 16:51:02.984234323 (95h24m57.415314454s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:51:03.289516404 (95h24m57.720596575s) [upload=99416]: Done uploading segments... Aug 12 16:51:03.289540049 (95h24m57.7206202s) [upload=99416]: Upload finished. err: <nil> Duration: 5.883304465s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......X..........X.........................................] duration: 2.013772469s FAIL (2025-08-12 17:11:18 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 12 16:39:09.008146755 (95h13m3.439226887s) [upload=99194]: Starting upload Aug 12 16:39:09.008201260 (95h13m3.439281391s) [upload=99194]: Done starting upload Aug 12 16:39:09.008258329 (95h13m3.43933846s) [upload=99194]: Uploading segments... Aug 12 16:39:09.008290200 (95h13m3.439370331s) [upload=99194]: Got next segment. Inline: true Aug 12 16:39:09.008295661 (95h13m3.439375782s) [upload=99194]: Waiting for error group managing segments... Aug 12 16:39:09.008328123 (95h13m3.439408254s) [upload=99194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:39:09.008334585 (95h13m3.439414716s) [upload=99194]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 16:39:09.008337160 (95h13m3.439417281s) [upload=99194]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:39:09.337170261 (95h13m3.768250392s) [upload=99194]: Done uploading segments... Aug 12 16:39:09.337190520 (95h13m3.768270651s) [upload=99194]: Upload finished. err: <nil> Duration:* 2.013799411s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.896750286s FAIL (2025-08-12 17:11:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 12 16:51:00.342944596 (95h24m54.774024727s) [upload=99416]: Starting upload Aug 12 16:51:00.343003058 (95h24m54.774083189s) [upload=99416]: Done starting upload Aug 12 16:51:00.343093682 (95h24m54.774173803s) [upload=99416]: Uploading segments... Aug 12 16:51:00.343214823 (95h24m54.774294954s) [upload=99416]: Got next segment. Inline: false Aug 12 16:51:00.343269027 (95h24m54.774349158s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:51:00.343273756 (95h24m54.774353887s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 16:51:00.585349151 (95h24m55.016429282s) [seg_pos={0 0}, upload=99416]: Begin upload segment... Aug 12 16:51:00.585396822 (95h24m55.016476973s) [seg_pos={0 0}, upload=99416]: Done begin upload segment. Aug 12 16:51:00.585410067 (95h24m55.016490208s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585507734 (95h24m55.016587895s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585886159 (95h24m55.0169663s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586089607 (95h24m55.017169738s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586541652 (95h24m55.017621803s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587004688 (95h24m55.018084819s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587619234 (95h24m55.018699385s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587983842 (95h24m55.019063983s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588303153 (95h24m55.019383284s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588688090 (95h24m55.019768221s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.589902393 (95h24m55.020982534s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.590674450 (95h24m55.021754601s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.591296961 (95h24m55.022377092s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.592189969 (95h24m55.02327011s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.593156619 (95h24m55.02423678s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.594056290 (95h24m55.025136421s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:02.362133336 (95h24m56.793213467s) [upload=99416]: Next returned nil segment Aug 12 16:51:02.362159547 (95h24m56.793239678s) [upload=99416]: Waiting for error group managing segments... Aug 12 16:51:02.650293798 (95h24m57.081373929s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.663772758 (95h24m57.094852919s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.696344749 (95h24m57.12742488s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.709104884 (95h24m57.140185025s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.722233484 (95h24m57.153313625s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.727048907 (95h24m57.158129038s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.741870376 (95h24m57.172950507s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.772732365 (95h24m57.203812507s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.805755481 (95h24m57.236835612s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.878280266 (95h24m57.309360397s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.913734542 (95h24m57.344814673s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.972369120 (95h24m57.403449251s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983815812 (95h24m57.414895943s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983842733 (95h24m57.414922864s) [seg_pos={0 0}, upload=99416]: Segment reached optimal threshold of 13 pieces. Aug 12 16:51:02.983942084 (95h24m57.415022215s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.983999583 (95h24m57.415079715s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984087211 (95h24m57.415167342s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984124363 (95h24m57.415204494s) [seg_pos={0 0}, upload=99416]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 12 16:51:02.984229683 (95h24m57.415309814s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 16:51:02.984234323 (95h24m57.415314454s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:51:03.289516404 (95h24m57.720596575s) [upload=99416]: Done uploading segments... Aug 12 16:51:03.289540049 (95h24m57.7206202s) [upload=99416]: Upload finished. err: <nil> Duration: 5.896772339s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [X...X.............................X.X.........X..........X..] duration: 6.16239153s FAIL (2025-08-12 17:10:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after b0982c89d4a14d25bd833df87489bff9fd295b7477379b6d824f265e357cc0f2 != before 197684ff454ac1a3a74c23a2c042188f74f051027843e0f6c54d853073e620af ======LOGS====== Aug 12 17:09:40.321853865 (95h43m34.752933996s) [upload=99722]: Starting upload Aug 12 17:09:40.321893551 (95h43m34.752973682s) [upload=99722]: Done starting upload Aug 12 17:09:40.321966250 (95h43m34.753046381s) [upload=99722]: Uploading segments... Aug 12 17:09:40.322074217 (95h43m34.753154348s) [upload=99722]: Got next segment. Inline: false Aug 12 17:09:40.322110977 (95h43m34.753191108s) [upload=99722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 17:09:40.322117018 (95h43m34.753197149s) [upload=99722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 17:09:40.774677309 (95h43m35.20575744s) [seg_pos={0 0}, upload=99722]: Begin upload segment... Aug 12 17:09:40.774728086 (95h43m35.205808217s) [seg_pos={0 0}, upload=99722]: Done begin upload segment. Aug 12 17:09:40.774751501 (95h43m35.205831622s) [noise=true , address=104.36.82.14:4233 , piece_id=G57MOVDGURNTF3GII4PG6INCDVLEFHQG4ZKECN4RDFQ3ZOFJJMPQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.774889235 (95h43m35.205969436s) [noise=true , address=104.36.82.34:4562 , piece_id=XSQXN4UZ4XALRCL2HC4SH63Q7VX6OFS63PILMNVLW6YUJ3SDZTQQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775115477 (95h43m35.206195608s) [noise=true , address=104.36.82.60:4358 , piece_id=6WK4MJI4MT6BWYXUQDPWKKEG6HVRHHN66IF27B3RCVA2K3IEEGEQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775394030 (95h43m35.206474161s) [noise=true , address=104.36.82.61:4388 , piece_id=MJL5EIV4QAGB57TYHAVPJWMRNJ2NV2XI3PDU2KG2NRUFOU5YX66Q, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775612628 (95h43m35.20669276s) [noise=true , address=104.36.82.20:4446 , piece_id=STM6GTU5C7KSEZKNEQDBBKG3KUESFA66XI4M7C5F2NDIOIMXRURA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775684887 (95h43m35.206765018s) [noise=true , address=45.91.116.5:4323 , piece_id=7OWV6OPJPRGM5MPZ25ULPBRADADRT5QMFXRJYNTDJNHJUC566WJA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775929696 (95h43m35.207009817s) [noise=true , address=79.127.201.215:4349 , piece_id=DWHXJZ6JYHNFZ7SYWLDBXKPPFEJ6NYASMN5CVBAVDGVYQURQ3PXA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776002796 (95h43m35.207082917s) [noise=true , address=104.36.82.7:4919 , piece_id=ARXPHRRZZ2OMTFTMUSHZFZ7ONGOZU3JEVUAAVZVFHOCUFBOJE6AA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776223618 (95h43m35.207303749s) [noise=true , address=104.36.82.46:4500 , piece_id=CRSOKFHEYK2G4NOSRZNJ3LYPJ5TLYAQHCRY2K36DTKKYYQQHRLDA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776298280 (95h43m35.207378401s) [noise=true , address=77.90.31.12:4623 , piece_id=HATORCXVWJQF5K2VFTDI52NIHMMWTDZDIMSFNGOSVFIIJARUJJWQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776500628 (95h43m35.207580759s) [noise=true , address=104.36.82.36:4624 , piece_id=EHDUBB4CI2FSOOV5WRWHLXRNSZOLS6NJVLZOWGNFNUKDASABC56A, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776585781 (95h43m35.207665912s) [noise=true , address=104.36.82.12:4159 , piece_id=U5JLWCP2MNCMXDEVPPESXRDJYKAYZQXIP2DQ3VDHUUNVSOPDWUJA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776747651 (95h43m35.207827782s) [noise=true , address=104.36.82.8:4995 , piece_id=3SD4HYUCN3Y7YAQP6IFRZKN2A2KD2YP2BDK3FDKRVEG3RTI6KQ4A, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776809178 (95h43m35.207889309s) [noise=true , address=104.36.82.58:4282 , piece_id=FKV7JWSXMZOBTOSEJIG4THDZVD4CTQNWESMC3AZG7UAEE7SQWUZA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.777003861 (95h43m35.208083992s) [noise=true , address=77.90.31.19:4865 , piece_id=KEEXRKILFITPIXVHQIJ5WEA3N5JWZR4SOYX2YKQXX5RIHAGPEAIA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.777030371 (95h43m35.208110503s) [noise=true , address=104.36.82.47:4524 , piece_id=VPOIP7WFL27O37X5JK4YOWMRC3ZROKBJYGLGJE2YPZNL7L7FXPKQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.777074015 (95h43m35.208154146s) [noise=true , address=104.36.82.17:4359 , piece_id=ZE6ZCIFIGRIRPCSTOXGYKICSMQ4MRW5EOVNNVD4X5WW2YOM6IYCA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.777244772 (95h43m35.208324903s) [noise=true , address=104.36.82.23:4475 , piece_id=2LS7X3RFXUFYRPCUNSJCANN2I6QYM2EEYWXEO7IN7GHKEYX45CWQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:42.295179902 (95h43m36.726260033s) [upload=99722]: Next returned nil segment Aug 12 17:09:42.295194640 (95h43m36.726274761s) [upload=99722]: Waiting for error group managing segments... Aug 12 17:09:42.729495544 (95h43m37.160575665s) [noise=true , address=45.91.116.5:4323 , piece_id=7OWV6OPJPRGM5MPZ25ULPBRADADRT5QMFXRJYNTDJNHJUC566WJA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.732790118 (95h43m37.163870289s) [noise=true , address=104.36.82.34:4562 , piece_id=XSQXN4UZ4XALRCL2HC4SH63Q7VX6OFS63PILMNVLW6YUJ3SDZTQQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.747195941 (95h43m37.178276072s) [noise=true , address=104.36.82.58:4282 , piece_id=FKV7JWSXMZOBTOSEJIG4THDZVD4CTQNWESMC3AZG7UAEE7SQWUZA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.754626518 (95h43m37.18570665s) [noise=true , address=104.36.82.7:4919 , piece_id=ARXPHRRZZ2OMTFTMUSHZFZ7ONGOZU3JEVUAAVZVFHOCUFBOJE6AA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.791808389 (95h43m37.22288852s) [noise=true , address=104.36.82.47:4524 , piece_id=VPOIP7WFL27O37X5JK4YOWMRC3ZROKBJYGLGJE2YPZNL7L7FXPKQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.803351946 (95h43m37.234432067s) [noise=true , address=104.36.82.20:4446 , piece_id=STM6GTU5C7KSEZKNEQDBBKG3KUESFA66XI4M7C5F2NDIOIMXRURA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.846122721 (95h43m37.277202852s) [noise=true , address=104.36.82.12:4159 , piece_id=U5JLWCP2MNCMXDEVPPESXRDJYKAYZQXIP2DQ3VDHUUNVSOPDWUJA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.856142241 (95h43m37.287222373s) [noise=true , address=104.36.82.14:4233 , piece_id=G57MOVDGURNTF3GII4PG6INCDVLEFHQG4ZKECN4RDFQ3ZOFJJMPQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.860973705 (95h43m37.292053836s) [noise=true , address=104.36.82.8:4995 , piece_id=3SD4HYUCN3Y7YAQP6IFRZKN2A2KD2YP2BDK3FDKRVEG3RTI6KQ4A, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.880185945 (95h43m37.311266076s) [noise=true , address=104.36.82.23:4475 , piece_id=2LS7X3RFXUFYRPCUNSJCANN2I6QYM2EEYWXEO7IN7GHKEYX45CWQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.881755528 (95h43m37.312835659s) [noise=true , address=104.36.82.17:4359 , piece_id=ZE6ZCIFIGRIRPCSTOXGYKICSMQ4MRW5EOVNNVD4X5WW2YOM6IYCA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.889261359 (95h43m37.32034149s) [noise=true , address=104.36.82.46:4500 , piece_id=CRSOKFHEYK2G4NOSRZNJ3LYPJ5TLYAQHCRY2K36DTKKYYQQHRLDA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.943274955 (95h43m37.374355116s) [noise=true , address=104.36.82.61:4388 , piece_id=MJL5EIV4QAGB57TYHAVPJWMRNJ2NV2XI3PDU2KG2NRUFOU5YX66Q, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.952686322 (95h43m37.383766453s) [noise=true , address=104.36.82.36:4624 , piece_id=EHDUBB4CI2FSOOV5WRWHLXRNSZOLS6NJVLZOWGNFNUKDASABC56A, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.952709246 (95h43m37.383789377s) [seg_pos={0 0}, upload=99722]: Segment reached optimal threshold of 14 pieces. Aug 12 17:09:42.952895372 (95h43m37.383975504s) [noise=true , address=77.90.31.19:4865 , piece_id=KEEXRKILFITPIXVHQIJ5WEA3N5JWZR4SOYX2YKQXX5RIHAGPEAIA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bguBghQonV5sTHaWteyUEnpp5KNzBKbvBpbvrUNFDCveHN77X): protocol: expected piece hash; context canceled; EOF Aug 12 17:09:42.952962120 (95h43m37.384042251s) [noise=true , address=79.127.201.215:4349 , piece_id=DWHXJZ6JYHNFZ7SYWLDBXKPPFEJ6NYASMN5CVBAVDGVYQURQ3PXA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y1KYQJsvbNxfySU7mXySUmJtEDVAE7fst1XVRHU5N8Txmjn5T): protocol: expected piece hash; context canceled; EOF Aug 12 17:09:42.952984222 (95h43m37.384064353s) [noise=true , address=77.90.31.12:4623 , piece_id=HATORCXVWJQF5K2VFTDI52NIHMMWTDZDIMSFNGOSVFIIJARUJJWQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: expected piece hash; context canceled; EOF Aug 12 17:09:42.953037214 (95h43m37.384117345s) [noise=true , address=104.36.82.60:4358 , piece_id=6WK4MJI4MT6BWYXUQDPWKKEG6HVRHHN66IF27B3RCVA2K3IEEGEQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dwCNEcA9o51ZN3UaN3Frxdawwf8DXBj86iQMywjW8XP4ESmw1): protocol: write tcp 188.166.225.170:43124->104.36.82.60:4358: use of closed network connection; write tcp 188.166.225.170:43124->104.36.82.60:4358: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43124->104.36.82.60:4358: use of closed network connection Aug 12 17:09:42.953048665 (95h43m37.384128786s) [seg_pos={0 0}, upload=99722]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 12 17:09:42.953101797 (95h43m37.384181928s) [upload=99722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 17:09:42.953105504 (95h43m37.384185635s) [upload=99722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 17:09:43.248448404 (95h43m37.679528535s) [upload=99722]: Done uploading segments... Aug 12 17:09:43.248466649 (95h43m37.67954677s) [upload=99722]: Upload finished. err: <nil> Duration: 6.162425966s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.X...X.............................X.X.........X..........X.] duration: 5.724616245s FAIL (2025-08-12 17:10:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after b0982c89d4a14d25bd833df87489bff9fd295b7477379b6d824f265e357cc0f2 != before 197684ff454ac1a3a74c23a2c042188f74f051027843e0f6c54d853073e620af ======LOGS====== Aug 12 17:09:40.321853865 (95h43m34.752933996s) [upload=99722]: Starting upload Aug 12 17:09:40.321893551 (95h43m34.752973682s) [upload=99722]: Done starting upload Aug 12 17:09:40.321966250 (95h43m34.753046381s) [upload=99722]: Uploading segments... Aug 12 17:09:40.322074217 (95h43m34.753154348s) [upload=99722]: Got next segment. Inline: false Aug 12 17:09:40.322110977 (95h43m34.753191108s) [upload=99722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 17:09:40.322117018 (95h43m34.753197149s) [upload=99722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 17:09:40.774677309 (95h43m35.20575744s) [seg_pos={0 0}, upload=99722]: Begin upload segment... Aug 12 17:09:40.774728086 (95h43m35.205808217s) [seg_pos={0 0}, upload=99722]: Done begin upload segment. Aug 12 17:09:40.774751501 (95h43m35.205831622s) [noise=true , address=104.36.82.14:4233 , piece_id=G57MOVDGURNTF3GII4PG6INCDVLEFHQG4ZKECN4RDFQ3ZOFJJMPQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.774889235 (95h43m35.205969436s) [noise=true , address=104.36.82.34:4562 , piece_id=XSQXN4UZ4XALRCL2HC4SH63Q7VX6OFS63PILMNVLW6YUJ3SDZTQQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775115477 (95h43m35.206195608s) [noise=true , address=104.36.82.60:4358 , piece_id=6WK4MJI4MT6BWYXUQDPWKKEG6HVRHHN66IF27B3RCVA2K3IEEGEQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775394030 (95h43m35.206474161s) [noise=true , address=104.36.82.61:4388 , piece_id=MJL5EIV4QAGB57TYHAVPJWMRNJ2NV2XI3PDU2KG2NRUFOU5YX66Q, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775612628 (95h43m35.20669276s) [noise=true , address=104.36.82.20:4446 , piece_id=STM6GTU5C7KSEZKNEQDBBKG3KUESFA66XI4M7C5F2NDIOIMXRURA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775684887 (95h43m35.206765018s) [noise=true , address=45.91.116.5:4323 , piece_id=7OWV6OPJPRGM5MPZ25ULPBRADADRT5QMFXRJYNTDJNHJUC566WJA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775929696 (95h43m35.207009817s) [noise=true , address=79.127.201.215:4349 , piece_id=DWHXJZ6JYHNFZ7SYWLDBXKPPFEJ6NYASMN5CVBAVDGVYQURQ3PXA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776002796 (95h43m35.207082917s) [noise=true , address=104.36.82.7:4919 , piece_id=ARXPHRRZZ2OMTFTMUSHZFZ7ONGOZU3JEVUAAVZVFHOCUFBOJE6AA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776223618 (95h43m35.207303749s) [noise=true , address=104.36.82.46:4500 , piece_id=CRSOKFHEYK2G4NOSRZNJ3LYPJ5TLYAQHCRY2K36DTKKYYQQHRLDA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776298280 (95h43m35.207378401s) [noise=true , address=77.90.31.12:4623 , piece_id=HATORCXVWJQF5K2VFTDI52NIHMMWTDZDIMSFNGOSVFIIJARUJJWQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776500628 (95h43m35.207580759s) [noise=true , address=104.36.82.36:4624 , piece_id=EHDUBB4CI2FSOOV5WRWHLXRNSZOLS6NJVLZOWGNFNUKDASABC56A, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776585781 (95h43m35.207665912s) [noise=true , address=104.36.82.12:4159 , piece_id=U5JLWCP2MNCMXDEVPPESXRDJYKAYZQXIP2DQ3VDHUUNVSOPDWUJA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776747651 (95h43m35.207827782s) [noise=true , address=104.36.82.8:4995 , piece_id=3SD4HYUCN3Y7YAQP6IFRZKN2A2KD2YP2BDK3FDKRVEG3RTI6KQ4A, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776809178 (95h43m35.207889309s) [noise=true , address=104.36.82.58:4282 , piece_id=FKV7JWSXMZOBTOSEJIG4THDZVD4CTQNWESMC3AZG7UAEE7SQWUZA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.777003861 (95h43m35.208083992s) [noise=true , address=77.90.31.19:4865 , piece_id=KEEXRKILFITPIXVHQIJ5WEA3N5JWZR4SOYX2YKQXX5RIHAGPEAIA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.777030371 (95h43m35.208110503s) [noise=true , address=104.36.82.47:4524 , piece_id=VPOIP7WFL27O37X5JK4YOWMRC3ZROKBJYGLGJE2YPZNL7L7FXPKQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.777074015 (95h43m35.208154146s) [noise=true , address=104.36.82.17:4359 , piece_id=ZE6ZCIFIGRIRPCSTOXGYKICSMQ4MRW5EOVNNVD4X5WW2YOM6IYCA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.777244772 (95h43m35.208324903s) [noise=true , address=104.36.82.23:4475 , piece_id=2LS7X3RFXUFYRPCUNSJCANN2I6QYM2EEYWXEO7IN7GHKEYX45CWQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:42.295179902 (95h43m36.726260033s) [upload=99722]: Next returned nil segment Aug 12 17:09:42.295194640 (95h43m36.726274761s) [upload=99722]: Waiting for error group managing segments... Aug 12 17:09:42.729495544 (95h43m37.160575665s) [noise=true , address=45.91.116.5:4323 , piece_id=7OWV6OPJPRGM5MPZ25ULPBRADADRT5QMFXRJYNTDJNHJUC566WJA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.732790118 (95h43m37.163870289s) [noise=true , address=104.36.82.34:4562 , piece_id=XSQXN4UZ4XALRCL2HC4SH63Q7VX6OFS63PILMNVLW6YUJ3SDZTQQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.747195941 (95h43m37.178276072s) [noise=true , address=104.36.82.58:4282 , piece_id=FKV7JWSXMZOBTOSEJIG4THDZVD4CTQNWESMC3AZG7UAEE7SQWUZA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.754626518 (95h43m37.18570665s) [noise=true , address=104.36.82.7:4919 , piece_id=ARXPHRRZZ2OMTFTMUSHZFZ7ONGOZU3JEVUAAVZVFHOCUFBOJE6AA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.791808389 (95h43m37.22288852s) [noise=true , address=104.36.82.47:4524 , piece_id=VPOIP7WFL27O37X5JK4YOWMRC3ZROKBJYGLGJE2YPZNL7L7FXPKQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.803351946 (95h43m37.234432067s) [noise=true , address=104.36.82.20:4446 , piece_id=STM6GTU5C7KSEZKNEQDBBKG3KUESFA66XI4M7C5F2NDIOIMXRURA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.846122721 (95h43m37.277202852s) [noise=true , address=104.36.82.12:4159 , piece_id=U5JLWCP2MNCMXDEVPPESXRDJYKAYZQXIP2DQ3VDHUUNVSOPDWUJA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.856142241 (95h43m37.287222373s) [noise=true , address=104.36.82.14:4233 , piece_id=G57MOVDGURNTF3GII4PG6INCDVLEFHQG4ZKECN4RDFQ3ZOFJJMPQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.860973705 (95h43m37.292053836s) [noise=true , address=104.36.82.8:4995 , piece_id=3SD4HYUCN3Y7YAQP6IFRZKN2A2KD2YP2BDK3FDKRVEG3RTI6KQ4A, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.880185945 (95h43m37.311266076s) [noise=true , address=104.36.82.23:4475 , piece_id=2LS7X3RFXUFYRPCUNSJCANN2I6QYM2EEYWXEO7IN7GHKEYX45CWQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.881755528 (95h43m37.312835659s) [noise=true , address=104.36.82.17:4359 , piece_id=ZE6ZCIFIGRIRPCSTOXGYKICSMQ4MRW5EOVNNVD4X5WW2YOM6IYCA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.889261359 (95h43m37.32034149s) [noise=true , address=104.36.82.46:4500 , piece_id=CRSOKFHEYK2G4NOSRZNJ3LYPJ5TLYAQHCRY2K36DTKKYYQQHRLDA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.943274955 (95h43m37.374355116s) [noise=true , address=104.36.82.61:4388 , piece_id=MJL5EIV4QAGB57TYHAVPJWMRNJ2NV2XI3PDU2KG2NRUFOU5YX66Q, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.952686322 (95h43m37.383766453s) [noise=true , address=104.36.82.36:4624 , piece_id=EHDUBB4CI2FSOOV5WRWHLXRNSZOLS6NJVLZOWGNFNUKDASABC56A, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.952709246 (95h43m37.383789377s) [seg_pos={0 0}, upload=99722]: Segment reached optimal threshold of 14 pieces. Aug 12 17:09:42.952895372 (95h43m37.383975504s) [noise=true , address=77.90.31.19:4865 , piece_id=KEEXRKILFITPIXVHQIJ5WEA3N5JWZR4SOYX2YKQXX5RIHAGPEAIA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bguBghQonV5sTHaWteyUEnpp5KNzBKbvBpbvrUNFDCveHN77X): protocol: expected piece hash; context canceled; EOF Aug 12 17:09:42.952962120 (95h43m37.384042251s) [noise=true , address=79.127.201.215:4349 , piece_id=DWHXJZ6JYHNFZ7SYWLDBXKPPFEJ6NYASMN5CVBAVDGVYQURQ3PXA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y1KYQJsvbNxfySU7mXySUmJtEDVAE7fst1XVRHU5N8Txmjn5T): protocol: expected piece hash; context canceled; EOF Aug 12 17:09:42.952984222 (95h43m37.384064353s) [noise=true , address=77.90.31.12:4623 , piece_id=HATORCXVWJQF5K2VFTDI52NIHMMWTDZDIMSFNGOSVFIIJARUJJWQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: expected piece hash; context canceled; EOF Aug 12 17:09:42.953037214 (95h43m37.384117345s) [noise=true , address=104.36.82.60:4358 , piece_id=6WK4MJI4MT6BWYXUQDPWKKEG6HVRHHN66IF27B3RCVA2K3IEEGEQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dwCNEcA9o51ZN3UaN3Frxdawwf8DXBj86iQMywjW8XP4ESmw1): protocol: write tcp 188.166.225.170:43124->104.36.82.60:4358: use of closed network connection; write tcp 188.166.225.170:43124->104.36.82.60:4358: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43124->104.36.82.60:4358: use of closed network connection Aug 12 17:09:42.953048665 (95h43m37.384128786s) [seg_pos={0 0}, upload=99722]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 12 17:09:42.953101797 (95h43m37.384181928s) [upload=99722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 17:09:42.953105504 (95h43m37.384185635s) [upload=99722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 17:09:43.248448404 (95h43m37.679528535s) [upload=99722]: Done uploading segments... Aug 12 17:09:43.248466649 (95h43m37.67954677s) [upload=99722]: Upload finished. err: <nil> Duration: 5.724643238s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........X..........X........................................] duration: 1.879153567s FAIL (2025-08-12 17:10:17 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 12 16:39:09.008146755 (95h13m3.439226887s) [upload=99194]: Starting upload Aug 12 16:39:09.008201260 (95h13m3.439281391s) [upload=99194]: Done starting upload Aug 12 16:39:09.008258329 (95h13m3.43933846s) [upload=99194]: Uploading segments... Aug 12 16:39:09.008290200 (95h13m3.439370331s) [upload=99194]: Got next segment. Inline: true Aug 12 16:39:09.008295661 (95h13m3.439375782s) [upload=99194]: Waiting for error group managing segments... Aug 12 16:39:09.008328123 (95h13m3.439408254s) [upload=99194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:39:09.008334585 (95h13m3.439414716s) [upload=99194]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 16:39:09.008337160 (95h13m3.439417281s) [upload=99194]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:39:09.337170261 (95h13m3.768250392s) [upload=99194]: Done uploading segments... Aug 12 16:39:09.337190520 (95h13m3.768270651s) [upload=99194]: Upload finished. err: <nil> Duration:* 1.879179858s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.8038749s FAIL (2025-08-12 17:09:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 12 16:51:00.342944596 (95h24m54.774024727s) [upload=99416]: Starting upload Aug 12 16:51:00.343003058 (95h24m54.774083189s) [upload=99416]: Done starting upload Aug 12 16:51:00.343093682 (95h24m54.774173803s) [upload=99416]: Uploading segments... Aug 12 16:51:00.343214823 (95h24m54.774294954s) [upload=99416]: Got next segment. Inline: false Aug 12 16:51:00.343269027 (95h24m54.774349158s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:51:00.343273756 (95h24m54.774353887s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 16:51:00.585349151 (95h24m55.016429282s) [seg_pos={0 0}, upload=99416]: Begin upload segment... Aug 12 16:51:00.585396822 (95h24m55.016476973s) [seg_pos={0 0}, upload=99416]: Done begin upload segment. Aug 12 16:51:00.585410067 (95h24m55.016490208s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585507734 (95h24m55.016587895s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585886159 (95h24m55.0169663s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586089607 (95h24m55.017169738s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586541652 (95h24m55.017621803s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587004688 (95h24m55.018084819s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587619234 (95h24m55.018699385s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587983842 (95h24m55.019063983s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588303153 (95h24m55.019383284s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588688090 (95h24m55.019768221s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.589902393 (95h24m55.020982534s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.590674450 (95h24m55.021754601s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.591296961 (95h24m55.022377092s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.592189969 (95h24m55.02327011s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.593156619 (95h24m55.02423678s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.594056290 (95h24m55.025136421s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:02.362133336 (95h24m56.793213467s) [upload=99416]: Next returned nil segment Aug 12 16:51:02.362159547 (95h24m56.793239678s) [upload=99416]: Waiting for error group managing segments... Aug 12 16:51:02.650293798 (95h24m57.081373929s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.663772758 (95h24m57.094852919s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.696344749 (95h24m57.12742488s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.709104884 (95h24m57.140185025s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.722233484 (95h24m57.153313625s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.727048907 (95h24m57.158129038s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.741870376 (95h24m57.172950507s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.772732365 (95h24m57.203812507s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.805755481 (95h24m57.236835612s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.878280266 (95h24m57.309360397s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.913734542 (95h24m57.344814673s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.972369120 (95h24m57.403449251s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983815812 (95h24m57.414895943s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983842733 (95h24m57.414922864s) [seg_pos={0 0}, upload=99416]: Segment reached optimal threshold of 13 pieces. Aug 12 16:51:02.983942084 (95h24m57.415022215s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.983999583 (95h24m57.415079715s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984087211 (95h24m57.415167342s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984124363 (95h24m57.415204494s) [seg_pos={0 0}, upload=99416]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 12 16:51:02.984229683 (95h24m57.415309814s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 16:51:02.984234323 (95h24m57.415314454s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:51:03.289516404 (95h24m57.720596575s) [upload=99416]: Done uploading segments... Aug 12 16:51:03.289540049 (95h24m57.7206202s) [upload=99416]: Upload finished. err: <nil> Duration: 5.803902403s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [X.X...X.............................X.X.........X..........X] duration: 5.928134411s FAIL (2025-08-12 17:09:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after b0982c89d4a14d25bd833df87489bff9fd295b7477379b6d824f265e357cc0f2 != before 197684ff454ac1a3a74c23a2c042188f74f051027843e0f6c54d853073e620af ======LOGS====== Aug 12 17:09:40.321853865 (95h43m34.752933996s) [upload=99722]: Starting upload Aug 12 17:09:40.321893551 (95h43m34.752973682s) [upload=99722]: Done starting upload Aug 12 17:09:40.321966250 (95h43m34.753046381s) [upload=99722]: Uploading segments... Aug 12 17:09:40.322074217 (95h43m34.753154348s) [upload=99722]: Got next segment. Inline: false Aug 12 17:09:40.322110977 (95h43m34.753191108s) [upload=99722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 17:09:40.322117018 (95h43m34.753197149s) [upload=99722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 17:09:40.774677309 (95h43m35.20575744s) [seg_pos={0 0}, upload=99722]: Begin upload segment... Aug 12 17:09:40.774728086 (95h43m35.205808217s) [seg_pos={0 0}, upload=99722]: Done begin upload segment. Aug 12 17:09:40.774751501 (95h43m35.205831622s) [noise=true , address=104.36.82.14:4233 , piece_id=G57MOVDGURNTF3GII4PG6INCDVLEFHQG4ZKECN4RDFQ3ZOFJJMPQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.774889235 (95h43m35.205969436s) [noise=true , address=104.36.82.34:4562 , piece_id=XSQXN4UZ4XALRCL2HC4SH63Q7VX6OFS63PILMNVLW6YUJ3SDZTQQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775115477 (95h43m35.206195608s) [noise=true , address=104.36.82.60:4358 , piece_id=6WK4MJI4MT6BWYXUQDPWKKEG6HVRHHN66IF27B3RCVA2K3IEEGEQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775394030 (95h43m35.206474161s) [noise=true , address=104.36.82.61:4388 , piece_id=MJL5EIV4QAGB57TYHAVPJWMRNJ2NV2XI3PDU2KG2NRUFOU5YX66Q, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775612628 (95h43m35.20669276s) [noise=true , address=104.36.82.20:4446 , piece_id=STM6GTU5C7KSEZKNEQDBBKG3KUESFA66XI4M7C5F2NDIOIMXRURA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775684887 (95h43m35.206765018s) [noise=true , address=45.91.116.5:4323 , piece_id=7OWV6OPJPRGM5MPZ25ULPBRADADRT5QMFXRJYNTDJNHJUC566WJA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.775929696 (95h43m35.207009817s) [noise=true , address=79.127.201.215:4349 , piece_id=DWHXJZ6JYHNFZ7SYWLDBXKPPFEJ6NYASMN5CVBAVDGVYQURQ3PXA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776002796 (95h43m35.207082917s) [noise=true , address=104.36.82.7:4919 , piece_id=ARXPHRRZZ2OMTFTMUSHZFZ7ONGOZU3JEVUAAVZVFHOCUFBOJE6AA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776223618 (95h43m35.207303749s) [noise=true , address=104.36.82.46:4500 , piece_id=CRSOKFHEYK2G4NOSRZNJ3LYPJ5TLYAQHCRY2K36DTKKYYQQHRLDA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776298280 (95h43m35.207378401s) [noise=true , address=77.90.31.12:4623 , piece_id=HATORCXVWJQF5K2VFTDI52NIHMMWTDZDIMSFNGOSVFIIJARUJJWQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776500628 (95h43m35.207580759s) [noise=true , address=104.36.82.36:4624 , piece_id=EHDUBB4CI2FSOOV5WRWHLXRNSZOLS6NJVLZOWGNFNUKDASABC56A, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776585781 (95h43m35.207665912s) [noise=true , address=104.36.82.12:4159 , piece_id=U5JLWCP2MNCMXDEVPPESXRDJYKAYZQXIP2DQ3VDHUUNVSOPDWUJA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776747651 (95h43m35.207827782s) [noise=true , address=104.36.82.8:4995 , piece_id=3SD4HYUCN3Y7YAQP6IFRZKN2A2KD2YP2BDK3FDKRVEG3RTI6KQ4A, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.776809178 (95h43m35.207889309s) [noise=true , address=104.36.82.58:4282 , piece_id=FKV7JWSXMZOBTOSEJIG4THDZVD4CTQNWESMC3AZG7UAEE7SQWUZA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.777003861 (95h43m35.208083992s) [noise=true , address=77.90.31.19:4865 , piece_id=KEEXRKILFITPIXVHQIJ5WEA3N5JWZR4SOYX2YKQXX5RIHAGPEAIA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.777030371 (95h43m35.208110503s) [noise=true , address=104.36.82.47:4524 , piece_id=VPOIP7WFL27O37X5JK4YOWMRC3ZROKBJYGLGJE2YPZNL7L7FXPKQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.777074015 (95h43m35.208154146s) [noise=true , address=104.36.82.17:4359 , piece_id=ZE6ZCIFIGRIRPCSTOXGYKICSMQ4MRW5EOVNNVD4X5WW2YOM6IYCA, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:40.777244772 (95h43m35.208324903s) [noise=true , address=104.36.82.23:4475 , piece_id=2LS7X3RFXUFYRPCUNSJCANN2I6QYM2EEYWXEO7IN7GHKEYX45CWQ, seg_pos={0 0}, upload=99722]: Uploading piece... Aug 12 17:09:42.295179902 (95h43m36.726260033s) [upload=99722]: Next returned nil segment Aug 12 17:09:42.295194640 (95h43m36.726274761s) [upload=99722]: Waiting for error group managing segments... Aug 12 17:09:42.729495544 (95h43m37.160575665s) [noise=true , address=45.91.116.5:4323 , piece_id=7OWV6OPJPRGM5MPZ25ULPBRADADRT5QMFXRJYNTDJNHJUC566WJA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.732790118 (95h43m37.163870289s) [noise=true , address=104.36.82.34:4562 , piece_id=XSQXN4UZ4XALRCL2HC4SH63Q7VX6OFS63PILMNVLW6YUJ3SDZTQQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.747195941 (95h43m37.178276072s) [noise=true , address=104.36.82.58:4282 , piece_id=FKV7JWSXMZOBTOSEJIG4THDZVD4CTQNWESMC3AZG7UAEE7SQWUZA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.754626518 (95h43m37.18570665s) [noise=true , address=104.36.82.7:4919 , piece_id=ARXPHRRZZ2OMTFTMUSHZFZ7ONGOZU3JEVUAAVZVFHOCUFBOJE6AA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.791808389 (95h43m37.22288852s) [noise=true , address=104.36.82.47:4524 , piece_id=VPOIP7WFL27O37X5JK4YOWMRC3ZROKBJYGLGJE2YPZNL7L7FXPKQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.803351946 (95h43m37.234432067s) [noise=true , address=104.36.82.20:4446 , piece_id=STM6GTU5C7KSEZKNEQDBBKG3KUESFA66XI4M7C5F2NDIOIMXRURA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.846122721 (95h43m37.277202852s) [noise=true , address=104.36.82.12:4159 , piece_id=U5JLWCP2MNCMXDEVPPESXRDJYKAYZQXIP2DQ3VDHUUNVSOPDWUJA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.856142241 (95h43m37.287222373s) [noise=true , address=104.36.82.14:4233 , piece_id=G57MOVDGURNTF3GII4PG6INCDVLEFHQG4ZKECN4RDFQ3ZOFJJMPQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.860973705 (95h43m37.292053836s) [noise=true , address=104.36.82.8:4995 , piece_id=3SD4HYUCN3Y7YAQP6IFRZKN2A2KD2YP2BDK3FDKRVEG3RTI6KQ4A, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.880185945 (95h43m37.311266076s) [noise=true , address=104.36.82.23:4475 , piece_id=2LS7X3RFXUFYRPCUNSJCANN2I6QYM2EEYWXEO7IN7GHKEYX45CWQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.881755528 (95h43m37.312835659s) [noise=true , address=104.36.82.17:4359 , piece_id=ZE6ZCIFIGRIRPCSTOXGYKICSMQ4MRW5EOVNNVD4X5WW2YOM6IYCA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.889261359 (95h43m37.32034149s) [noise=true , address=104.36.82.46:4500 , piece_id=CRSOKFHEYK2G4NOSRZNJ3LYPJ5TLYAQHCRY2K36DTKKYYQQHRLDA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.943274955 (95h43m37.374355116s) [noise=true , address=104.36.82.61:4388 , piece_id=MJL5EIV4QAGB57TYHAVPJWMRNJ2NV2XI3PDU2KG2NRUFOU5YX66Q, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.952686322 (95h43m37.383766453s) [noise=true , address=104.36.82.36:4624 , piece_id=EHDUBB4CI2FSOOV5WRWHLXRNSZOLS6NJVLZOWGNFNUKDASABC56A, seg_pos={0 0}, upload=99722]: Done uploading piece. err: <nil> Aug 12 17:09:42.952709246 (95h43m37.383789377s) [seg_pos={0 0}, upload=99722]: Segment reached optimal threshold of 14 pieces. Aug 12 17:09:42.952895372 (95h43m37.383975504s) [noise=true , address=77.90.31.19:4865 , piece_id=KEEXRKILFITPIXVHQIJ5WEA3N5JWZR4SOYX2YKQXX5RIHAGPEAIA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bguBghQonV5sTHaWteyUEnpp5KNzBKbvBpbvrUNFDCveHN77X): protocol: expected piece hash; context canceled; EOF Aug 12 17:09:42.952962120 (95h43m37.384042251s) [noise=true , address=79.127.201.215:4349 , piece_id=DWHXJZ6JYHNFZ7SYWLDBXKPPFEJ6NYASMN5CVBAVDGVYQURQ3PXA, seg_pos={0 0}, upload=99722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y1KYQJsvbNxfySU7mXySUmJtEDVAE7fst1XVRHU5N8Txmjn5T): protocol: expected piece hash; context canceled; EOF Aug 12 17:09:42.952984222 (95h43m37.384064353s) [noise=true , address=77.90.31.12:4623 , piece_id=HATORCXVWJQF5K2VFTDI52NIHMMWTDZDIMSFNGOSVFIIJARUJJWQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ANjFmZrb9SDcqEHW8CBzeMtwvmceoLH9MtPabAjBw8wXjBuaM): protocol: expected piece hash; context canceled; EOF Aug 12 17:09:42.953037214 (95h43m37.384117345s) [noise=true , address=104.36.82.60:4358 , piece_id=6WK4MJI4MT6BWYXUQDPWKKEG6HVRHHN66IF27B3RCVA2K3IEEGEQ, seg_pos={0 0}, upload=99722]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dwCNEcA9o51ZN3UaN3Frxdawwf8DXBj86iQMywjW8XP4ESmw1): protocol: write tcp 188.166.225.170:43124->104.36.82.60:4358: use of closed network connection; write tcp 188.166.225.170:43124->104.36.82.60:4358: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43124->104.36.82.60:4358: use of closed network connection Aug 12 17:09:42.953048665 (95h43m37.384128786s) [seg_pos={0 0}, upload=99722]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 12 17:09:42.953101797 (95h43m37.384181928s) [upload=99722]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 17:09:42.953105504 (95h43m37.384185635s) [upload=99722]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 17:09:43.248448404 (95h43m37.679528535s) [upload=99722]: Done uploading segments... Aug 12 17:09:43.248466649 (95h43m37.67954677s) [upload=99722]: Upload finished. err: <nil> Duration: 5.92815981s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........X..........X.......................................] duration: 1.881627821s FAIL (2025-08-12 17:09:36 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 12 16:39:09.008146755 (95h13m3.439226887s) [upload=99194]: Starting upload Aug 12 16:39:09.008201260 (95h13m3.439281391s) [upload=99194]: Done starting upload Aug 12 16:39:09.008258329 (95h13m3.43933846s) [upload=99194]: Uploading segments... Aug 12 16:39:09.008290200 (95h13m3.439370331s) [upload=99194]: Got next segment. Inline: true Aug 12 16:39:09.008295661 (95h13m3.439375782s) [upload=99194]: Waiting for error group managing segments... Aug 12 16:39:09.008328123 (95h13m3.439408254s) [upload=99194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:39:09.008334585 (95h13m3.439414716s) [upload=99194]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 16:39:09.008337160 (95h13m3.439417281s) [upload=99194]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:39:09.337170261 (95h13m3.768250392s) [upload=99194]: Done uploading segments... Aug 12 16:39:09.337190520 (95h13m3.768270651s) [upload=99194]: Upload finished. err: <nil> Duration:* 1.881654141s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.853667728s FAIL (2025-08-12 17:09:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 12 16:51:00.342944596 (95h24m54.774024727s) [upload=99416]: Starting upload Aug 12 16:51:00.343003058 (95h24m54.774083189s) [upload=99416]: Done starting upload Aug 12 16:51:00.343093682 (95h24m54.774173803s) [upload=99416]: Uploading segments... Aug 12 16:51:00.343214823 (95h24m54.774294954s) [upload=99416]: Got next segment. Inline: false Aug 12 16:51:00.343269027 (95h24m54.774349158s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:51:00.343273756 (95h24m54.774353887s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 16:51:00.585349151 (95h24m55.016429282s) [seg_pos={0 0}, upload=99416]: Begin upload segment... Aug 12 16:51:00.585396822 (95h24m55.016476973s) [seg_pos={0 0}, upload=99416]: Done begin upload segment. Aug 12 16:51:00.585410067 (95h24m55.016490208s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585507734 (95h24m55.016587895s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585886159 (95h24m55.0169663s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586089607 (95h24m55.017169738s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586541652 (95h24m55.017621803s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587004688 (95h24m55.018084819s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587619234 (95h24m55.018699385s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587983842 (95h24m55.019063983s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588303153 (95h24m55.019383284s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588688090 (95h24m55.019768221s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.589902393 (95h24m55.020982534s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.590674450 (95h24m55.021754601s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.591296961 (95h24m55.022377092s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.592189969 (95h24m55.02327011s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.593156619 (95h24m55.02423678s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.594056290 (95h24m55.025136421s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:02.362133336 (95h24m56.793213467s) [upload=99416]: Next returned nil segment Aug 12 16:51:02.362159547 (95h24m56.793239678s) [upload=99416]: Waiting for error group managing segments... Aug 12 16:51:02.650293798 (95h24m57.081373929s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.663772758 (95h24m57.094852919s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.696344749 (95h24m57.12742488s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.709104884 (95h24m57.140185025s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.722233484 (95h24m57.153313625s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.727048907 (95h24m57.158129038s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.741870376 (95h24m57.172950507s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.772732365 (95h24m57.203812507s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.805755481 (95h24m57.236835612s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.878280266 (95h24m57.309360397s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.913734542 (95h24m57.344814673s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.972369120 (95h24m57.403449251s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983815812 (95h24m57.414895943s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983842733 (95h24m57.414922864s) [seg_pos={0 0}, upload=99416]: Segment reached optimal threshold of 13 pieces. Aug 12 16:51:02.983942084 (95h24m57.415022215s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.983999583 (95h24m57.415079715s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984087211 (95h24m57.415167342s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984124363 (95h24m57.415204494s) [seg_pos={0 0}, upload=99416]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 12 16:51:02.984229683 (95h24m57.415309814s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 16:51:02.984234323 (95h24m57.415314454s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:51:03.289516404 (95h24m57.720596575s) [upload=99416]: Done uploading segments... Aug 12 16:51:03.289540049 (95h24m57.7206202s) [upload=99416]: Upload finished. err: <nil> Duration: 5.853701411s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.X.X...X.............................X.X.........X..........] duration: 6.054569819s FAIL (2025-08-12 17:09:02 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 12 16:59:38.687530140 (95h33m33.118610271s) [upload=99560]: Starting upload Aug 12 16:59:38.687574635 (95h33m33.118654766s) [upload=99560]: Done starting upload Aug 12 16:59:38.687634360 (95h33m33.118714491s) [upload=99560]: Uploading segments... Aug 12 16:59:38.687738799 (95h33m33.11881892s) [upload=99560]: Got next segment. Inline: false Aug 12 16:59:38.687772675 (95h33m33.118852806s) [upload=99560]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:59:38.687777594 (95h33m33.118857725s) [upload=99560]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 16:59:39.197902742 (95h33m33.628982873s) [seg_pos={0 0}, upload=99560]: Begin upload segment... Aug 12 16:59:39.197949550 (95h33m33.629029682s) [seg_pos={0 0}, upload=99560]: Done begin upload segment. Aug 12 16:59:39.197963598 (95h33m33.629043729s) [noise=true , address=104.36.82.21:4495 , piece_id=6PW2LL4YAWQO6EGJWRPGO7PDK7VRJ63Y6E5FIS3FIMQDQS37JE2Q, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.197979778 (95h33m33.629059909s) [noise=true , address=104.36.82.61:4405 , piece_id=42XK2XF4LW6JDERRITW4BGU4SASVH5XAIAI4I7H7WJBMPDD2UVZQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198019174 (95h33m33.629099305s) [noise=true , address=104.36.82.62:4410 , piece_id=NNHLCIVHJSFAQOF5BPARODUZ7NDKRIO7VZ4TXMDPSIT7CPBM7LGA, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198062236 (95h33m33.629142357s) [noise=true , address=77.90.31.19:4844 , piece_id=6ACO5XJCM7JKXI6DZNS7Q6FMNIIF2H4QX5GFFZP275LGK4LAG6KQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198334707 (95h33m33.629414828s) [noise=true , address=104.36.82.58:4285 , piece_id=RSNIOADYVBK6XD3XNUUNRLYBSDKJNLXTGGQSUQA5MF2KOGCY7WVQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198339377 (95h33m33.629419508s) [noise=true , address=79.127.201.214:4320 , piece_id=KTDAMSYK7CDMKJDYD7EL45LDWFRECADQYJWINN6BFJUKFCHCXKCQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198607269 (95h33m33.6296874s) [noise=true , address=104.36.82.23:4467 , piece_id=72OG4FRL66Q522MHHPWUEX5F7TBW7BHTNBUEB2FTLVX6GNEKEBUQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198865934 (95h33m33.629946065s) [noise=true , address=104.36.82.35:4607 , piece_id=MEGUKOCNGAXTQF6JKS4UM2FTUOJNH22XUUCEOLZG56XXBFRESSRQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.200169648 (95h33m33.631249779s) [noise=true , address=104.36.82.35:4583 , piece_id=F425SAQ7MSRGMEAFG75XFNNVQPJOXKKXOCF4DXUYB3ZJJX7O3Y2Q, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.200909183 (95h33m33.631989315s) [noise=true , address=104.36.82.3:4796 , piece_id=GKGL3BVJKJI34563P6NAAMQBH6K53WQ2XR5A5TRPTA4VK24WN7HA, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.201550832 (95h33m33.632630963s) [noise=true , address=104.36.82.47:4519 , piece_id=CWQ2QS6LJLG2MGBY3B5KHVUKDJQ64ZAWHCQSEIMBHYXVZ2KQONWA, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.202192539 (95h33m33.63327267s) [noise=true , address=193.149.164.143:4698 , piece_id=I7BDBG7QK66PCZNNU3MDNT3IENTGDLSMHIQMFDAJGGGLTWNNW7IA, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.202627732 (95h33m33.633707863s) [noise=true , address=104.36.82.5:4869 , piece_id=TOHQG3BR5YJQNJDMVDWUZ6DLBKNOI5NCOCOI6G2L44BSJ24CR42Q, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.202954337 (95h33m33.634034468s) [noise=true , address=104.36.82.46:4490 , piece_id=MFHE7SH22T6KPVQD57THJ47O7ANA3UL6DSXPXE5VKKZGZHCXHWMQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.203178796 (95h33m33.634258927s) [noise=true , address=104.36.82.18:4366 , piece_id=ABVSGRVBMLWFCYO4RGTYAZ4XMYFCKKL4OMFAMS2WLD4OFNCHKUZQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.203380622 (95h33m33.634460753s) [noise=true , address=104.36.82.16:4297 , piece_id=VDDGO3TNUEPV45LB3J65UZX7WYF4BORGTQ5FZDLOLPUIWGHFDG6Q, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.203821095 (95h33m33.634901226s) [noise=true , address=79.127.201.215:4350 , piece_id=J5FYRE2SDV7OCGN2OK3BBY2TZ6JKCJ5TUIPJ67SEMTQRBIJN3E2A, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.205764854 (95h33m33.636844995s) [noise=true , address=104.36.82.60:4372 , piece_id=M54ATKGBFCQWGXWTCFU7GDGGWPTN5UJONPFDT6QEW4BP5CJCRKIA, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:40.363639120 (95h33m34.794719261s) [upload=99560]: Next returned nil segment Aug 12 16:59:40.363657104 (95h33m34.794737235s) [upload=99560]: Waiting for error group managing segments... Aug 12 16:59:40.692247382 (95h33m35.123327513s) [noise=true , address=193.149.164.143:4698 , piece_id=I7BDBG7QK66PCZNNU3MDNT3IENTGDLSMHIQMFDAJGGGLTWNNW7IA, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.169955785 (95h33m35.601035916s) [noise=true , address=104.36.82.21:4495 , piece_id=6PW2LL4YAWQO6EGJWRPGO7PDK7VRJ63Y6E5FIS3FIMQDQS37JE2Q, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.173959115 (95h33m35.605039236s) [noise=true , address=77.90.31.19:4844 , piece_id=6ACO5XJCM7JKXI6DZNS7Q6FMNIIF2H4QX5GFFZP275LGK4LAG6KQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.202640181 (95h33m35.633720302s) [noise=true , address=104.36.82.62:4410 , piece_id=NNHLCIVHJSFAQOF5BPARODUZ7NDKRIO7VZ4TXMDPSIT7CPBM7LGA, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.217521444 (95h33m35.648601575s) [noise=true , address=104.36.82.47:4519 , piece_id=CWQ2QS6LJLG2MGBY3B5KHVUKDJQ64ZAWHCQSEIMBHYXVZ2KQONWA, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.246049799 (95h33m35.67712993s) [noise=true , address=104.36.82.3:4796 , piece_id=GKGL3BVJKJI34563P6NAAMQBH6K53WQ2XR5A5TRPTA4VK24WN7HA, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.264431349 (95h33m35.69551148s) [noise=true , address=104.36.82.18:4366 , piece_id=ABVSGRVBMLWFCYO4RGTYAZ4XMYFCKKL4OMFAMS2WLD4OFNCHKUZQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.367834365 (95h33m35.798914496s) [noise=true , address=104.36.82.60:4372 , piece_id=M54ATKGBFCQWGXWTCFU7GDGGWPTN5UJONPFDT6QEW4BP5CJCRKIA, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.368115543 (95h33m35.799195664s) [noise=true , address=104.36.82.58:4285 , piece_id=RSNIOADYVBK6XD3XNUUNRLYBSDKJNLXTGGQSUQA5MF2KOGCY7WVQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.396228793 (95h33m35.827308934s) [noise=true , address=104.36.82.46:4490 , piece_id=MFHE7SH22T6KPVQD57THJ47O7ANA3UL6DSXPXE5VKKZGZHCXHWMQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.404358458 (95h33m35.835438599s) [noise=true , address=104.36.82.35:4583 , piece_id=F425SAQ7MSRGMEAFG75XFNNVQPJOXKKXOCF4DXUYB3ZJJX7O3Y2Q, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.424749884 (95h33m35.855830015s) [noise=true , address=104.36.82.16:4297 , piece_id=VDDGO3TNUEPV45LB3J65UZX7WYF4BORGTQ5FZDLOLPUIWGHFDG6Q, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.457148824 (95h33m35.888228945s) [noise=true , address=104.36.82.35:4607 , piece_id=MEGUKOCNGAXTQF6JKS4UM2FTUOJNH22XUUCEOLZG56XXBFRESSRQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.505612983 (95h33m35.936693114s) [noise=true , address=104.36.82.61:4405 , piece_id=42XK2XF4LW6JDERRITW4BGU4SASVH5XAIAI4I7H7WJBMPDD2UVZQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.505632861 (95h33m35.936712992s) [seg_pos={0 0}, upload=99560]: Segment reached optimal threshold of 14 pieces. Aug 12 16:59:41.505886516 (95h33m35.936966647s) [noise=true , address=79.127.201.214:4320 , piece_id=KTDAMSYK7CDMKJDYD7EL45LDWFRECADQYJWINN6BFJUKFCHCXKCQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZLCXYP6sppXZcmj49C8jjgXbLGzgyowtpb1um9XZguzfVnM2r): protocol: expected piece hash; context canceled; EOF Aug 12 16:59:41.505927656 (95h33m35.937007787s) [noise=true , address=104.36.82.23:4467 , piece_id=72OG4FRL66Q522MHHPWUEX5F7TBW7BHTNBUEB2FTLVX6GNEKEBUQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RegmCcGuZED5cPuhVJ9mLRuouCHBEZ3hecf7aseHsMCgo1ajT): protocol: expected piece hash; context canceled; EOF Aug 12 16:59:41.505990275 (95h33m35.937070406s) [noise=true , address=79.127.201.215:4350 , piece_id=J5FYRE2SDV7OCGN2OK3BBY2TZ6JKCJ5TUIPJ67SEMTQRBIJN3E2A, seg_pos={0 0}, upload=99560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WhZAWvHCTj5xVkUuVZtgkFB1dhFKpBPq18YMKkepFx5qxkUAr): protocol: expected piece hash; context canceled; EOF Aug 12 16:59:41.506052915 (95h33m35.937133036s) [noise=true , address=104.36.82.5:4869 , piece_id=TOHQG3BR5YJQNJDMVDWUZ6DLBKNOI5NCOCOI6G2L44BSJ24CR42Q, seg_pos={0 0}, upload=99560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1273igSPBbnYnP5bTMPjCphwkCSK4WdBdBMFVgfnrPaq4oSUZQr): protocol: write tcp 188.166.225.170:39306->104.36.82.5:4869: use of closed network connection; write tcp 188.166.225.170:39306->104.36.82.5:4869: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39306->104.36.82.5:4869: use of closed network connection Aug 12 16:59:41.506075017 (95h33m35.937155148s) [seg_pos={0 0}, upload=99560]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 12 16:59:41.506137476 (95h33m35.937217597s) [upload=99560]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 16:59:41.506141796 (95h33m35.937221927s) [upload=99560]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:59:41.812022822 (95h33m36.243102943s) [upload=99560]: Done uploading segments... Aug 12 16:59:41.812034574 (95h33m36.243114695s) [upload=99560]: Upload finished. err: <nil> Duration: 6.054593264s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.911065746s FAIL (2025-08-12 17:08:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 12 16:51:00.342944596 (95h24m54.774024727s) [upload=99416]: Starting upload Aug 12 16:51:00.343003058 (95h24m54.774083189s) [upload=99416]: Done starting upload Aug 12 16:51:00.343093682 (95h24m54.774173803s) [upload=99416]: Uploading segments... Aug 12 16:51:00.343214823 (95h24m54.774294954s) [upload=99416]: Got next segment. Inline: false Aug 12 16:51:00.343269027 (95h24m54.774349158s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:51:00.343273756 (95h24m54.774353887s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 16:51:00.585349151 (95h24m55.016429282s) [seg_pos={0 0}, upload=99416]: Begin upload segment... Aug 12 16:51:00.585396822 (95h24m55.016476973s) [seg_pos={0 0}, upload=99416]: Done begin upload segment. Aug 12 16:51:00.585410067 (95h24m55.016490208s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585507734 (95h24m55.016587895s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585886159 (95h24m55.0169663s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586089607 (95h24m55.017169738s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586541652 (95h24m55.017621803s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587004688 (95h24m55.018084819s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587619234 (95h24m55.018699385s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587983842 (95h24m55.019063983s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588303153 (95h24m55.019383284s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588688090 (95h24m55.019768221s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.589902393 (95h24m55.020982534s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.590674450 (95h24m55.021754601s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.591296961 (95h24m55.022377092s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.592189969 (95h24m55.02327011s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.593156619 (95h24m55.02423678s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.594056290 (95h24m55.025136421s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:02.362133336 (95h24m56.793213467s) [upload=99416]: Next returned nil segment Aug 12 16:51:02.362159547 (95h24m56.793239678s) [upload=99416]: Waiting for error group managing segments... Aug 12 16:51:02.650293798 (95h24m57.081373929s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.663772758 (95h24m57.094852919s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.696344749 (95h24m57.12742488s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.709104884 (95h24m57.140185025s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.722233484 (95h24m57.153313625s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.727048907 (95h24m57.158129038s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.741870376 (95h24m57.172950507s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.772732365 (95h24m57.203812507s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.805755481 (95h24m57.236835612s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.878280266 (95h24m57.309360397s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.913734542 (95h24m57.344814673s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.972369120 (95h24m57.403449251s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983815812 (95h24m57.414895943s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983842733 (95h24m57.414922864s) [seg_pos={0 0}, upload=99416]: Segment reached optimal threshold of 13 pieces. Aug 12 16:51:02.983942084 (95h24m57.415022215s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.983999583 (95h24m57.415079715s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984087211 (95h24m57.415167342s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984124363 (95h24m57.415204494s) [seg_pos={0 0}, upload=99416]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 12 16:51:02.984229683 (95h24m57.415309814s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 16:51:02.984234323 (95h24m57.415314454s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:51:03.289516404 (95h24m57.720596575s) [upload=99416]: Done uploading segments... Aug 12 16:51:03.289540049 (95h24m57.7206202s) [upload=99416]: Upload finished. err: <nil> Duration: 5.911093771s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [..X.X...X.............................X.X.........X.........] duration: 5.728352738s FAIL (2025-08-12 17:08:38 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 12 16:59:38.687530140 (95h33m33.118610271s) [upload=99560]: Starting upload Aug 12 16:59:38.687574635 (95h33m33.118654766s) [upload=99560]: Done starting upload Aug 12 16:59:38.687634360 (95h33m33.118714491s) [upload=99560]: Uploading segments... Aug 12 16:59:38.687738799 (95h33m33.11881892s) [upload=99560]: Got next segment. Inline: false Aug 12 16:59:38.687772675 (95h33m33.118852806s) [upload=99560]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:59:38.687777594 (95h33m33.118857725s) [upload=99560]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 16:59:39.197902742 (95h33m33.628982873s) [seg_pos={0 0}, upload=99560]: Begin upload segment... Aug 12 16:59:39.197949550 (95h33m33.629029682s) [seg_pos={0 0}, upload=99560]: Done begin upload segment. Aug 12 16:59:39.197963598 (95h33m33.629043729s) [noise=true , address=104.36.82.21:4495 , piece_id=6PW2LL4YAWQO6EGJWRPGO7PDK7VRJ63Y6E5FIS3FIMQDQS37JE2Q, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.197979778 (95h33m33.629059909s) [noise=true , address=104.36.82.61:4405 , piece_id=42XK2XF4LW6JDERRITW4BGU4SASVH5XAIAI4I7H7WJBMPDD2UVZQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198019174 (95h33m33.629099305s) [noise=true , address=104.36.82.62:4410 , piece_id=NNHLCIVHJSFAQOF5BPARODUZ7NDKRIO7VZ4TXMDPSIT7CPBM7LGA, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198062236 (95h33m33.629142357s) [noise=true , address=77.90.31.19:4844 , piece_id=6ACO5XJCM7JKXI6DZNS7Q6FMNIIF2H4QX5GFFZP275LGK4LAG6KQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198334707 (95h33m33.629414828s) [noise=true , address=104.36.82.58:4285 , piece_id=RSNIOADYVBK6XD3XNUUNRLYBSDKJNLXTGGQSUQA5MF2KOGCY7WVQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198339377 (95h33m33.629419508s) [noise=true , address=79.127.201.214:4320 , piece_id=KTDAMSYK7CDMKJDYD7EL45LDWFRECADQYJWINN6BFJUKFCHCXKCQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198607269 (95h33m33.6296874s) [noise=true , address=104.36.82.23:4467 , piece_id=72OG4FRL66Q522MHHPWUEX5F7TBW7BHTNBUEB2FTLVX6GNEKEBUQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198865934 (95h33m33.629946065s) [noise=true , address=104.36.82.35:4607 , piece_id=MEGUKOCNGAXTQF6JKS4UM2FTUOJNH22XUUCEOLZG56XXBFRESSRQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.200169648 (95h33m33.631249779s) [noise=true , address=104.36.82.35:4583 , piece_id=F425SAQ7MSRGMEAFG75XFNNVQPJOXKKXOCF4DXUYB3ZJJX7O3Y2Q, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.200909183 (95h33m33.631989315s) [noise=true , address=104.36.82.3:4796 , piece_id=GKGL3BVJKJI34563P6NAAMQBH6K53WQ2XR5A5TRPTA4VK24WN7HA, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.201550832 (95h33m33.632630963s) [noise=true , address=104.36.82.47:4519 , piece_id=CWQ2QS6LJLG2MGBY3B5KHVUKDJQ64ZAWHCQSEIMBHYXVZ2KQONWA, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.202192539 (95h33m33.63327267s) [noise=true , address=193.149.164.143:4698 , piece_id=I7BDBG7QK66PCZNNU3MDNT3IENTGDLSMHIQMFDAJGGGLTWNNW7IA, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.202627732 (95h33m33.633707863s) [noise=true , address=104.36.82.5:4869 , piece_id=TOHQG3BR5YJQNJDMVDWUZ6DLBKNOI5NCOCOI6G2L44BSJ24CR42Q, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.202954337 (95h33m33.634034468s) [noise=true , address=104.36.82.46:4490 , piece_id=MFHE7SH22T6KPVQD57THJ47O7ANA3UL6DSXPXE5VKKZGZHCXHWMQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.203178796 (95h33m33.634258927s) [noise=true , address=104.36.82.18:4366 , piece_id=ABVSGRVBMLWFCYO4RGTYAZ4XMYFCKKL4OMFAMS2WLD4OFNCHKUZQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.203380622 (95h33m33.634460753s) [noise=true , address=104.36.82.16:4297 , piece_id=VDDGO3TNUEPV45LB3J65UZX7WYF4BORGTQ5FZDLOLPUIWGHFDG6Q, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.203821095 (95h33m33.634901226s) [noise=true , address=79.127.201.215:4350 , piece_id=J5FYRE2SDV7OCGN2OK3BBY2TZ6JKCJ5TUIPJ67SEMTQRBIJN3E2A, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.205764854 (95h33m33.636844995s) [noise=true , address=104.36.82.60:4372 , piece_id=M54ATKGBFCQWGXWTCFU7GDGGWPTN5UJONPFDT6QEW4BP5CJCRKIA, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:40.363639120 (95h33m34.794719261s) [upload=99560]: Next returned nil segment Aug 12 16:59:40.363657104 (95h33m34.794737235s) [upload=99560]: Waiting for error group managing segments... Aug 12 16:59:40.692247382 (95h33m35.123327513s) [noise=true , address=193.149.164.143:4698 , piece_id=I7BDBG7QK66PCZNNU3MDNT3IENTGDLSMHIQMFDAJGGGLTWNNW7IA, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.169955785 (95h33m35.601035916s) [noise=true , address=104.36.82.21:4495 , piece_id=6PW2LL4YAWQO6EGJWRPGO7PDK7VRJ63Y6E5FIS3FIMQDQS37JE2Q, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.173959115 (95h33m35.605039236s) [noise=true , address=77.90.31.19:4844 , piece_id=6ACO5XJCM7JKXI6DZNS7Q6FMNIIF2H4QX5GFFZP275LGK4LAG6KQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.202640181 (95h33m35.633720302s) [noise=true , address=104.36.82.62:4410 , piece_id=NNHLCIVHJSFAQOF5BPARODUZ7NDKRIO7VZ4TXMDPSIT7CPBM7LGA, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.217521444 (95h33m35.648601575s) [noise=true , address=104.36.82.47:4519 , piece_id=CWQ2QS6LJLG2MGBY3B5KHVUKDJQ64ZAWHCQSEIMBHYXVZ2KQONWA, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.246049799 (95h33m35.67712993s) [noise=true , address=104.36.82.3:4796 , piece_id=GKGL3BVJKJI34563P6NAAMQBH6K53WQ2XR5A5TRPTA4VK24WN7HA, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.264431349 (95h33m35.69551148s) [noise=true , address=104.36.82.18:4366 , piece_id=ABVSGRVBMLWFCYO4RGTYAZ4XMYFCKKL4OMFAMS2WLD4OFNCHKUZQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.367834365 (95h33m35.798914496s) [noise=true , address=104.36.82.60:4372 , piece_id=M54ATKGBFCQWGXWTCFU7GDGGWPTN5UJONPFDT6QEW4BP5CJCRKIA, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.368115543 (95h33m35.799195664s) [noise=true , address=104.36.82.58:4285 , piece_id=RSNIOADYVBK6XD3XNUUNRLYBSDKJNLXTGGQSUQA5MF2KOGCY7WVQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.396228793 (95h33m35.827308934s) [noise=true , address=104.36.82.46:4490 , piece_id=MFHE7SH22T6KPVQD57THJ47O7ANA3UL6DSXPXE5VKKZGZHCXHWMQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.404358458 (95h33m35.835438599s) [noise=true , address=104.36.82.35:4583 , piece_id=F425SAQ7MSRGMEAFG75XFNNVQPJOXKKXOCF4DXUYB3ZJJX7O3Y2Q, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.424749884 (95h33m35.855830015s) [noise=true , address=104.36.82.16:4297 , piece_id=VDDGO3TNUEPV45LB3J65UZX7WYF4BORGTQ5FZDLOLPUIWGHFDG6Q, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.457148824 (95h33m35.888228945s) [noise=true , address=104.36.82.35:4607 , piece_id=MEGUKOCNGAXTQF6JKS4UM2FTUOJNH22XUUCEOLZG56XXBFRESSRQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.505612983 (95h33m35.936693114s) [noise=true , address=104.36.82.61:4405 , piece_id=42XK2XF4LW6JDERRITW4BGU4SASVH5XAIAI4I7H7WJBMPDD2UVZQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.505632861 (95h33m35.936712992s) [seg_pos={0 0}, upload=99560]: Segment reached optimal threshold of 14 pieces. Aug 12 16:59:41.505886516 (95h33m35.936966647s) [noise=true , address=79.127.201.214:4320 , piece_id=KTDAMSYK7CDMKJDYD7EL45LDWFRECADQYJWINN6BFJUKFCHCXKCQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZLCXYP6sppXZcmj49C8jjgXbLGzgyowtpb1um9XZguzfVnM2r): protocol: expected piece hash; context canceled; EOF Aug 12 16:59:41.505927656 (95h33m35.937007787s) [noise=true , address=104.36.82.23:4467 , piece_id=72OG4FRL66Q522MHHPWUEX5F7TBW7BHTNBUEB2FTLVX6GNEKEBUQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RegmCcGuZED5cPuhVJ9mLRuouCHBEZ3hecf7aseHsMCgo1ajT): protocol: expected piece hash; context canceled; EOF Aug 12 16:59:41.505990275 (95h33m35.937070406s) [noise=true , address=79.127.201.215:4350 , piece_id=J5FYRE2SDV7OCGN2OK3BBY2TZ6JKCJ5TUIPJ67SEMTQRBIJN3E2A, seg_pos={0 0}, upload=99560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WhZAWvHCTj5xVkUuVZtgkFB1dhFKpBPq18YMKkepFx5qxkUAr): protocol: expected piece hash; context canceled; EOF Aug 12 16:59:41.506052915 (95h33m35.937133036s) [noise=true , address=104.36.82.5:4869 , piece_id=TOHQG3BR5YJQNJDMVDWUZ6DLBKNOI5NCOCOI6G2L44BSJ24CR42Q, seg_pos={0 0}, upload=99560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1273igSPBbnYnP5bTMPjCphwkCSK4WdBdBMFVgfnrPaq4oSUZQr): protocol: write tcp 188.166.225.170:39306->104.36.82.5:4869: use of closed network connection; write tcp 188.166.225.170:39306->104.36.82.5:4869: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39306->104.36.82.5:4869: use of closed network connection Aug 12 16:59:41.506075017 (95h33m35.937155148s) [seg_pos={0 0}, upload=99560]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 12 16:59:41.506137476 (95h33m35.937217597s) [upload=99560]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 16:59:41.506141796 (95h33m35.937221927s) [upload=99560]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:59:41.812022822 (95h33m36.243102943s) [upload=99560]: Done uploading segments... Aug 12 16:59:41.812034574 (95h33m36.243114695s) [upload=99560]: Upload finished. err: <nil> Duration: 5.72838008s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........X..........X......................................] duration: 1.823248441s FAIL (2025-08-12 17:08:35 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 12 16:39:09.008146755 (95h13m3.439226887s) [upload=99194]: Starting upload Aug 12 16:39:09.008201260 (95h13m3.439281391s) [upload=99194]: Done starting upload Aug 12 16:39:09.008258329 (95h13m3.43933846s) [upload=99194]: Uploading segments... Aug 12 16:39:09.008290200 (95h13m3.439370331s) [upload=99194]: Got next segment. Inline: true Aug 12 16:39:09.008295661 (95h13m3.439375782s) [upload=99194]: Waiting for error group managing segments... Aug 12 16:39:09.008328123 (95h13m3.439408254s) [upload=99194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:39:09.008334585 (95h13m3.439414716s) [upload=99194]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 16:39:09.008337160 (95h13m3.439417281s) [upload=99194]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:39:09.337170261 (95h13m3.768250392s) [upload=99194]: Done uploading segments... Aug 12 16:39:09.337190520 (95h13m3.768270651s) [upload=99194]: Upload finished. err: <nil> Duration:* 1.823276384s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........X..........X.....................................] duration: 1.941141189s FAIL (2025-08-12 17:08:05 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 12 16:39:09.008146755 (95h13m3.439226887s) [upload=99194]: Starting upload Aug 12 16:39:09.008201260 (95h13m3.439281391s) [upload=99194]: Done starting upload Aug 12 16:39:09.008258329 (95h13m3.43933846s) [upload=99194]: Uploading segments... Aug 12 16:39:09.008290200 (95h13m3.439370331s) [upload=99194]: Got next segment. Inline: true Aug 12 16:39:09.008295661 (95h13m3.439375782s) [upload=99194]: Waiting for error group managing segments... Aug 12 16:39:09.008328123 (95h13m3.439408254s) [upload=99194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:39:09.008334585 (95h13m3.439414716s) [upload=99194]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 16:39:09.008337160 (95h13m3.439417281s) [upload=99194]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:39:09.337170261 (95h13m3.768250392s) [upload=99194]: Done uploading segments... Aug 12 16:39:09.337190520 (95h13m3.768270651s) [upload=99194]: Upload finished. err: <nil> Duration:* 1.941166387s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.800802165s FAIL (2025-08-12 17:07:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 12 16:51:00.342944596 (95h24m54.774024727s) [upload=99416]: Starting upload Aug 12 16:51:00.343003058 (95h24m54.774083189s) [upload=99416]: Done starting upload Aug 12 16:51:00.343093682 (95h24m54.774173803s) [upload=99416]: Uploading segments... Aug 12 16:51:00.343214823 (95h24m54.774294954s) [upload=99416]: Got next segment. Inline: false Aug 12 16:51:00.343269027 (95h24m54.774349158s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:51:00.343273756 (95h24m54.774353887s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 16:51:00.585349151 (95h24m55.016429282s) [seg_pos={0 0}, upload=99416]: Begin upload segment... Aug 12 16:51:00.585396822 (95h24m55.016476973s) [seg_pos={0 0}, upload=99416]: Done begin upload segment. Aug 12 16:51:00.585410067 (95h24m55.016490208s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585507734 (95h24m55.016587895s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585886159 (95h24m55.0169663s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586089607 (95h24m55.017169738s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586541652 (95h24m55.017621803s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587004688 (95h24m55.018084819s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587619234 (95h24m55.018699385s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587983842 (95h24m55.019063983s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588303153 (95h24m55.019383284s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588688090 (95h24m55.019768221s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.589902393 (95h24m55.020982534s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.590674450 (95h24m55.021754601s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.591296961 (95h24m55.022377092s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.592189969 (95h24m55.02327011s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.593156619 (95h24m55.02423678s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.594056290 (95h24m55.025136421s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:02.362133336 (95h24m56.793213467s) [upload=99416]: Next returned nil segment Aug 12 16:51:02.362159547 (95h24m56.793239678s) [upload=99416]: Waiting for error group managing segments... Aug 12 16:51:02.650293798 (95h24m57.081373929s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.663772758 (95h24m57.094852919s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.696344749 (95h24m57.12742488s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.709104884 (95h24m57.140185025s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.722233484 (95h24m57.153313625s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.727048907 (95h24m57.158129038s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.741870376 (95h24m57.172950507s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.772732365 (95h24m57.203812507s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.805755481 (95h24m57.236835612s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.878280266 (95h24m57.309360397s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.913734542 (95h24m57.344814673s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.972369120 (95h24m57.403449251s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983815812 (95h24m57.414895943s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983842733 (95h24m57.414922864s) [seg_pos={0 0}, upload=99416]: Segment reached optimal threshold of 13 pieces. Aug 12 16:51:02.983942084 (95h24m57.415022215s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.983999583 (95h24m57.415079715s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984087211 (95h24m57.415167342s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984124363 (95h24m57.415204494s) [seg_pos={0 0}, upload=99416]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 12 16:51:02.984229683 (95h24m57.415309814s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 16:51:02.984234323 (95h24m57.415314454s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:51:03.289516404 (95h24m57.720596575s) [upload=99416]: Done uploading segments... Aug 12 16:51:03.289540049 (95h24m57.7206202s) [upload=99416]: Upload finished. err: <nil> Duration: 5.800819437s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [...X.X...X.............................X.X.........X........] duration: 5.834300616s FAIL (2025-08-12 17:07:29 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 12 16:59:38.687530140 (95h33m33.118610271s) [upload=99560]: Starting upload Aug 12 16:59:38.687574635 (95h33m33.118654766s) [upload=99560]: Done starting upload Aug 12 16:59:38.687634360 (95h33m33.118714491s) [upload=99560]: Uploading segments... Aug 12 16:59:38.687738799 (95h33m33.11881892s) [upload=99560]: Got next segment. Inline: false Aug 12 16:59:38.687772675 (95h33m33.118852806s) [upload=99560]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:59:38.687777594 (95h33m33.118857725s) [upload=99560]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 16:59:39.197902742 (95h33m33.628982873s) [seg_pos={0 0}, upload=99560]: Begin upload segment... Aug 12 16:59:39.197949550 (95h33m33.629029682s) [seg_pos={0 0}, upload=99560]: Done begin upload segment. Aug 12 16:59:39.197963598 (95h33m33.629043729s) [noise=true , address=104.36.82.21:4495 , piece_id=6PW2LL4YAWQO6EGJWRPGO7PDK7VRJ63Y6E5FIS3FIMQDQS37JE2Q, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.197979778 (95h33m33.629059909s) [noise=true , address=104.36.82.61:4405 , piece_id=42XK2XF4LW6JDERRITW4BGU4SASVH5XAIAI4I7H7WJBMPDD2UVZQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198019174 (95h33m33.629099305s) [noise=true , address=104.36.82.62:4410 , piece_id=NNHLCIVHJSFAQOF5BPARODUZ7NDKRIO7VZ4TXMDPSIT7CPBM7LGA, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198062236 (95h33m33.629142357s) [noise=true , address=77.90.31.19:4844 , piece_id=6ACO5XJCM7JKXI6DZNS7Q6FMNIIF2H4QX5GFFZP275LGK4LAG6KQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198334707 (95h33m33.629414828s) [noise=true , address=104.36.82.58:4285 , piece_id=RSNIOADYVBK6XD3XNUUNRLYBSDKJNLXTGGQSUQA5MF2KOGCY7WVQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198339377 (95h33m33.629419508s) [noise=true , address=79.127.201.214:4320 , piece_id=KTDAMSYK7CDMKJDYD7EL45LDWFRECADQYJWINN6BFJUKFCHCXKCQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198607269 (95h33m33.6296874s) [noise=true , address=104.36.82.23:4467 , piece_id=72OG4FRL66Q522MHHPWUEX5F7TBW7BHTNBUEB2FTLVX6GNEKEBUQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198865934 (95h33m33.629946065s) [noise=true , address=104.36.82.35:4607 , piece_id=MEGUKOCNGAXTQF6JKS4UM2FTUOJNH22XUUCEOLZG56XXBFRESSRQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.200169648 (95h33m33.631249779s) [noise=true , address=104.36.82.35:4583 , piece_id=F425SAQ7MSRGMEAFG75XFNNVQPJOXKKXOCF4DXUYB3ZJJX7O3Y2Q, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.200909183 (95h33m33.631989315s) [noise=true , address=104.36.82.3:4796 , piece_id=GKGL3BVJKJI34563P6NAAMQBH6K53WQ2XR5A5TRPTA4VK24WN7HA, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.201550832 (95h33m33.632630963s) [noise=true , address=104.36.82.47:4519 , piece_id=CWQ2QS6LJLG2MGBY3B5KHVUKDJQ64ZAWHCQSEIMBHYXVZ2KQONWA, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.202192539 (95h33m33.63327267s) [noise=true , address=193.149.164.143:4698 , piece_id=I7BDBG7QK66PCZNNU3MDNT3IENTGDLSMHIQMFDAJGGGLTWNNW7IA, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.202627732 (95h33m33.633707863s) [noise=true , address=104.36.82.5:4869 , piece_id=TOHQG3BR5YJQNJDMVDWUZ6DLBKNOI5NCOCOI6G2L44BSJ24CR42Q, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.202954337 (95h33m33.634034468s) [noise=true , address=104.36.82.46:4490 , piece_id=MFHE7SH22T6KPVQD57THJ47O7ANA3UL6DSXPXE5VKKZGZHCXHWMQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.203178796 (95h33m33.634258927s) [noise=true , address=104.36.82.18:4366 , piece_id=ABVSGRVBMLWFCYO4RGTYAZ4XMYFCKKL4OMFAMS2WLD4OFNCHKUZQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.203380622 (95h33m33.634460753s) [noise=true , address=104.36.82.16:4297 , piece_id=VDDGO3TNUEPV45LB3J65UZX7WYF4BORGTQ5FZDLOLPUIWGHFDG6Q, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.203821095 (95h33m33.634901226s) [noise=true , address=79.127.201.215:4350 , piece_id=J5FYRE2SDV7OCGN2OK3BBY2TZ6JKCJ5TUIPJ67SEMTQRBIJN3E2A, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.205764854 (95h33m33.636844995s) [noise=true , address=104.36.82.60:4372 , piece_id=M54ATKGBFCQWGXWTCFU7GDGGWPTN5UJONPFDT6QEW4BP5CJCRKIA, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:40.363639120 (95h33m34.794719261s) [upload=99560]: Next returned nil segment Aug 12 16:59:40.363657104 (95h33m34.794737235s) [upload=99560]: Waiting for error group managing segments... Aug 12 16:59:40.692247382 (95h33m35.123327513s) [noise=true , address=193.149.164.143:4698 , piece_id=I7BDBG7QK66PCZNNU3MDNT3IENTGDLSMHIQMFDAJGGGLTWNNW7IA, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.169955785 (95h33m35.601035916s) [noise=true , address=104.36.82.21:4495 , piece_id=6PW2LL4YAWQO6EGJWRPGO7PDK7VRJ63Y6E5FIS3FIMQDQS37JE2Q, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.173959115 (95h33m35.605039236s) [noise=true , address=77.90.31.19:4844 , piece_id=6ACO5XJCM7JKXI6DZNS7Q6FMNIIF2H4QX5GFFZP275LGK4LAG6KQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.202640181 (95h33m35.633720302s) [noise=true , address=104.36.82.62:4410 , piece_id=NNHLCIVHJSFAQOF5BPARODUZ7NDKRIO7VZ4TXMDPSIT7CPBM7LGA, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.217521444 (95h33m35.648601575s) [noise=true , address=104.36.82.47:4519 , piece_id=CWQ2QS6LJLG2MGBY3B5KHVUKDJQ64ZAWHCQSEIMBHYXVZ2KQONWA, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.246049799 (95h33m35.67712993s) [noise=true , address=104.36.82.3:4796 , piece_id=GKGL3BVJKJI34563P6NAAMQBH6K53WQ2XR5A5TRPTA4VK24WN7HA, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.264431349 (95h33m35.69551148s) [noise=true , address=104.36.82.18:4366 , piece_id=ABVSGRVBMLWFCYO4RGTYAZ4XMYFCKKL4OMFAMS2WLD4OFNCHKUZQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.367834365 (95h33m35.798914496s) [noise=true , address=104.36.82.60:4372 , piece_id=M54ATKGBFCQWGXWTCFU7GDGGWPTN5UJONPFDT6QEW4BP5CJCRKIA, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.368115543 (95h33m35.799195664s) [noise=true , address=104.36.82.58:4285 , piece_id=RSNIOADYVBK6XD3XNUUNRLYBSDKJNLXTGGQSUQA5MF2KOGCY7WVQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.396228793 (95h33m35.827308934s) [noise=true , address=104.36.82.46:4490 , piece_id=MFHE7SH22T6KPVQD57THJ47O7ANA3UL6DSXPXE5VKKZGZHCXHWMQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.404358458 (95h33m35.835438599s) [noise=true , address=104.36.82.35:4583 , piece_id=F425SAQ7MSRGMEAFG75XFNNVQPJOXKKXOCF4DXUYB3ZJJX7O3Y2Q, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.424749884 (95h33m35.855830015s) [noise=true , address=104.36.82.16:4297 , piece_id=VDDGO3TNUEPV45LB3J65UZX7WYF4BORGTQ5FZDLOLPUIWGHFDG6Q, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.457148824 (95h33m35.888228945s) [noise=true , address=104.36.82.35:4607 , piece_id=MEGUKOCNGAXTQF6JKS4UM2FTUOJNH22XUUCEOLZG56XXBFRESSRQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.505612983 (95h33m35.936693114s) [noise=true , address=104.36.82.61:4405 , piece_id=42XK2XF4LW6JDERRITW4BGU4SASVH5XAIAI4I7H7WJBMPDD2UVZQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.505632861 (95h33m35.936712992s) [seg_pos={0 0}, upload=99560]: Segment reached optimal threshold of 14 pieces. Aug 12 16:59:41.505886516 (95h33m35.936966647s) [noise=true , address=79.127.201.214:4320 , piece_id=KTDAMSYK7CDMKJDYD7EL45LDWFRECADQYJWINN6BFJUKFCHCXKCQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZLCXYP6sppXZcmj49C8jjgXbLGzgyowtpb1um9XZguzfVnM2r): protocol: expected piece hash; context canceled; EOF Aug 12 16:59:41.505927656 (95h33m35.937007787s) [noise=true , address=104.36.82.23:4467 , piece_id=72OG4FRL66Q522MHHPWUEX5F7TBW7BHTNBUEB2FTLVX6GNEKEBUQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RegmCcGuZED5cPuhVJ9mLRuouCHBEZ3hecf7aseHsMCgo1ajT): protocol: expected piece hash; context canceled; EOF Aug 12 16:59:41.505990275 (95h33m35.937070406s) [noise=true , address=79.127.201.215:4350 , piece_id=J5FYRE2SDV7OCGN2OK3BBY2TZ6JKCJ5TUIPJ67SEMTQRBIJN3E2A, seg_pos={0 0}, upload=99560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WhZAWvHCTj5xVkUuVZtgkFB1dhFKpBPq18YMKkepFx5qxkUAr): protocol: expected piece hash; context canceled; EOF Aug 12 16:59:41.506052915 (95h33m35.937133036s) [noise=true , address=104.36.82.5:4869 , piece_id=TOHQG3BR5YJQNJDMVDWUZ6DLBKNOI5NCOCOI6G2L44BSJ24CR42Q, seg_pos={0 0}, upload=99560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1273igSPBbnYnP5bTMPjCphwkCSK4WdBdBMFVgfnrPaq4oSUZQr): protocol: write tcp 188.166.225.170:39306->104.36.82.5:4869: use of closed network connection; write tcp 188.166.225.170:39306->104.36.82.5:4869: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39306->104.36.82.5:4869: use of closed network connection Aug 12 16:59:41.506075017 (95h33m35.937155148s) [seg_pos={0 0}, upload=99560]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 12 16:59:41.506137476 (95h33m35.937217597s) [upload=99560]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 16:59:41.506141796 (95h33m35.937221927s) [upload=99560]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:59:41.812022822 (95h33m36.243102943s) [upload=99560]: Done uploading segments... Aug 12 16:59:41.812034574 (95h33m36.243114695s) [upload=99560]: Upload finished. err: <nil> Duration: 5.834325775s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............X..........X....................................] duration: 2.062437931s FAIL (2025-08-12 17:07:08 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 12 16:39:09.008146755 (95h13m3.439226887s) [upload=99194]: Starting upload Aug 12 16:39:09.008201260 (95h13m3.439281391s) [upload=99194]: Done starting upload Aug 12 16:39:09.008258329 (95h13m3.43933846s) [upload=99194]: Uploading segments... Aug 12 16:39:09.008290200 (95h13m3.439370331s) [upload=99194]: Got next segment. Inline: true Aug 12 16:39:09.008295661 (95h13m3.439375782s) [upload=99194]: Waiting for error group managing segments... Aug 12 16:39:09.008328123 (95h13m3.439408254s) [upload=99194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:39:09.008334585 (95h13m3.439414716s) [upload=99194]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 16:39:09.008337160 (95h13m3.439417281s) [upload=99194]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:39:09.337170261 (95h13m3.768250392s) [upload=99194]: Done uploading segments... Aug 12 16:39:09.337190520 (95h13m3.768270651s) [upload=99194]: Upload finished. err: <nil> Duration:* 2.062462308s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.831590515s FAIL (2025-08-12 17:06:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 12 16:51:00.342944596 (95h24m54.774024727s) [upload=99416]: Starting upload Aug 12 16:51:00.343003058 (95h24m54.774083189s) [upload=99416]: Done starting upload Aug 12 16:51:00.343093682 (95h24m54.774173803s) [upload=99416]: Uploading segments... Aug 12 16:51:00.343214823 (95h24m54.774294954s) [upload=99416]: Got next segment. Inline: false Aug 12 16:51:00.343269027 (95h24m54.774349158s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:51:00.343273756 (95h24m54.774353887s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 16:51:00.585349151 (95h24m55.016429282s) [seg_pos={0 0}, upload=99416]: Begin upload segment... Aug 12 16:51:00.585396822 (95h24m55.016476973s) [seg_pos={0 0}, upload=99416]: Done begin upload segment. Aug 12 16:51:00.585410067 (95h24m55.016490208s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585507734 (95h24m55.016587895s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585886159 (95h24m55.0169663s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586089607 (95h24m55.017169738s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586541652 (95h24m55.017621803s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587004688 (95h24m55.018084819s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587619234 (95h24m55.018699385s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587983842 (95h24m55.019063983s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588303153 (95h24m55.019383284s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588688090 (95h24m55.019768221s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.589902393 (95h24m55.020982534s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.590674450 (95h24m55.021754601s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.591296961 (95h24m55.022377092s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.592189969 (95h24m55.02327011s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.593156619 (95h24m55.02423678s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.594056290 (95h24m55.025136421s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:02.362133336 (95h24m56.793213467s) [upload=99416]: Next returned nil segment Aug 12 16:51:02.362159547 (95h24m56.793239678s) [upload=99416]: Waiting for error group managing segments... Aug 12 16:51:02.650293798 (95h24m57.081373929s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.663772758 (95h24m57.094852919s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.696344749 (95h24m57.12742488s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.709104884 (95h24m57.140185025s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.722233484 (95h24m57.153313625s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.727048907 (95h24m57.158129038s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.741870376 (95h24m57.172950507s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.772732365 (95h24m57.203812507s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.805755481 (95h24m57.236835612s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.878280266 (95h24m57.309360397s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.913734542 (95h24m57.344814673s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.972369120 (95h24m57.403449251s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983815812 (95h24m57.414895943s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983842733 (95h24m57.414922864s) [seg_pos={0 0}, upload=99416]: Segment reached optimal threshold of 13 pieces. Aug 12 16:51:02.983942084 (95h24m57.415022215s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.983999583 (95h24m57.415079715s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984087211 (95h24m57.415167342s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984124363 (95h24m57.415204494s) [seg_pos={0 0}, upload=99416]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 12 16:51:02.984229683 (95h24m57.415309814s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 16:51:02.984234323 (95h24m57.415314454s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:51:03.289516404 (95h24m57.720596575s) [upload=99416]: Done uploading segments... Aug 12 16:51:03.289540049 (95h24m57.7206202s) [upload=99416]: Upload finished. err: <nil> Duration: 5.831622135s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [....X.X...X.............................X.X.........X.......] duration: 5.749315083s FAIL (2025-08-12 17:06:19 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 12 16:59:38.687530140 (95h33m33.118610271s) [upload=99560]: Starting upload Aug 12 16:59:38.687574635 (95h33m33.118654766s) [upload=99560]: Done starting upload Aug 12 16:59:38.687634360 (95h33m33.118714491s) [upload=99560]: Uploading segments... Aug 12 16:59:38.687738799 (95h33m33.11881892s) [upload=99560]: Got next segment. Inline: false Aug 12 16:59:38.687772675 (95h33m33.118852806s) [upload=99560]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:59:38.687777594 (95h33m33.118857725s) [upload=99560]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 16:59:39.197902742 (95h33m33.628982873s) [seg_pos={0 0}, upload=99560]: Begin upload segment... Aug 12 16:59:39.197949550 (95h33m33.629029682s) [seg_pos={0 0}, upload=99560]: Done begin upload segment. Aug 12 16:59:39.197963598 (95h33m33.629043729s) [noise=true , address=104.36.82.21:4495 , piece_id=6PW2LL4YAWQO6EGJWRPGO7PDK7VRJ63Y6E5FIS3FIMQDQS37JE2Q, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.197979778 (95h33m33.629059909s) [noise=true , address=104.36.82.61:4405 , piece_id=42XK2XF4LW6JDERRITW4BGU4SASVH5XAIAI4I7H7WJBMPDD2UVZQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198019174 (95h33m33.629099305s) [noise=true , address=104.36.82.62:4410 , piece_id=NNHLCIVHJSFAQOF5BPARODUZ7NDKRIO7VZ4TXMDPSIT7CPBM7LGA, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198062236 (95h33m33.629142357s) [noise=true , address=77.90.31.19:4844 , piece_id=6ACO5XJCM7JKXI6DZNS7Q6FMNIIF2H4QX5GFFZP275LGK4LAG6KQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198334707 (95h33m33.629414828s) [noise=true , address=104.36.82.58:4285 , piece_id=RSNIOADYVBK6XD3XNUUNRLYBSDKJNLXTGGQSUQA5MF2KOGCY7WVQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198339377 (95h33m33.629419508s) [noise=true , address=79.127.201.214:4320 , piece_id=KTDAMSYK7CDMKJDYD7EL45LDWFRECADQYJWINN6BFJUKFCHCXKCQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198607269 (95h33m33.6296874s) [noise=true , address=104.36.82.23:4467 , piece_id=72OG4FRL66Q522MHHPWUEX5F7TBW7BHTNBUEB2FTLVX6GNEKEBUQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198865934 (95h33m33.629946065s) [noise=true , address=104.36.82.35:4607 , piece_id=MEGUKOCNGAXTQF6JKS4UM2FTUOJNH22XUUCEOLZG56XXBFRESSRQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.200169648 (95h33m33.631249779s) [noise=true , address=104.36.82.35:4583 , piece_id=F425SAQ7MSRGMEAFG75XFNNVQPJOXKKXOCF4DXUYB3ZJJX7O3Y2Q, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.200909183 (95h33m33.631989315s) [noise=true , address=104.36.82.3:4796 , piece_id=GKGL3BVJKJI34563P6NAAMQBH6K53WQ2XR5A5TRPTA4VK24WN7HA, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.201550832 (95h33m33.632630963s) [noise=true , address=104.36.82.47:4519 , piece_id=CWQ2QS6LJLG2MGBY3B5KHVUKDJQ64ZAWHCQSEIMBHYXVZ2KQONWA, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.202192539 (95h33m33.63327267s) [noise=true , address=193.149.164.143:4698 , piece_id=I7BDBG7QK66PCZNNU3MDNT3IENTGDLSMHIQMFDAJGGGLTWNNW7IA, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.202627732 (95h33m33.633707863s) [noise=true , address=104.36.82.5:4869 , piece_id=TOHQG3BR5YJQNJDMVDWUZ6DLBKNOI5NCOCOI6G2L44BSJ24CR42Q, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.202954337 (95h33m33.634034468s) [noise=true , address=104.36.82.46:4490 , piece_id=MFHE7SH22T6KPVQD57THJ47O7ANA3UL6DSXPXE5VKKZGZHCXHWMQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.203178796 (95h33m33.634258927s) [noise=true , address=104.36.82.18:4366 , piece_id=ABVSGRVBMLWFCYO4RGTYAZ4XMYFCKKL4OMFAMS2WLD4OFNCHKUZQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.203380622 (95h33m33.634460753s) [noise=true , address=104.36.82.16:4297 , piece_id=VDDGO3TNUEPV45LB3J65UZX7WYF4BORGTQ5FZDLOLPUIWGHFDG6Q, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.203821095 (95h33m33.634901226s) [noise=true , address=79.127.201.215:4350 , piece_id=J5FYRE2SDV7OCGN2OK3BBY2TZ6JKCJ5TUIPJ67SEMTQRBIJN3E2A, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.205764854 (95h33m33.636844995s) [noise=true , address=104.36.82.60:4372 , piece_id=M54ATKGBFCQWGXWTCFU7GDGGWPTN5UJONPFDT6QEW4BP5CJCRKIA, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:40.363639120 (95h33m34.794719261s) [upload=99560]: Next returned nil segment Aug 12 16:59:40.363657104 (95h33m34.794737235s) [upload=99560]: Waiting for error group managing segments... Aug 12 16:59:40.692247382 (95h33m35.123327513s) [noise=true , address=193.149.164.143:4698 , piece_id=I7BDBG7QK66PCZNNU3MDNT3IENTGDLSMHIQMFDAJGGGLTWNNW7IA, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.169955785 (95h33m35.601035916s) [noise=true , address=104.36.82.21:4495 , piece_id=6PW2LL4YAWQO6EGJWRPGO7PDK7VRJ63Y6E5FIS3FIMQDQS37JE2Q, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.173959115 (95h33m35.605039236s) [noise=true , address=77.90.31.19:4844 , piece_id=6ACO5XJCM7JKXI6DZNS7Q6FMNIIF2H4QX5GFFZP275LGK4LAG6KQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.202640181 (95h33m35.633720302s) [noise=true , address=104.36.82.62:4410 , piece_id=NNHLCIVHJSFAQOF5BPARODUZ7NDKRIO7VZ4TXMDPSIT7CPBM7LGA, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.217521444 (95h33m35.648601575s) [noise=true , address=104.36.82.47:4519 , piece_id=CWQ2QS6LJLG2MGBY3B5KHVUKDJQ64ZAWHCQSEIMBHYXVZ2KQONWA, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.246049799 (95h33m35.67712993s) [noise=true , address=104.36.82.3:4796 , piece_id=GKGL3BVJKJI34563P6NAAMQBH6K53WQ2XR5A5TRPTA4VK24WN7HA, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.264431349 (95h33m35.69551148s) [noise=true , address=104.36.82.18:4366 , piece_id=ABVSGRVBMLWFCYO4RGTYAZ4XMYFCKKL4OMFAMS2WLD4OFNCHKUZQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.367834365 (95h33m35.798914496s) [noise=true , address=104.36.82.60:4372 , piece_id=M54ATKGBFCQWGXWTCFU7GDGGWPTN5UJONPFDT6QEW4BP5CJCRKIA, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.368115543 (95h33m35.799195664s) [noise=true , address=104.36.82.58:4285 , piece_id=RSNIOADYVBK6XD3XNUUNRLYBSDKJNLXTGGQSUQA5MF2KOGCY7WVQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.396228793 (95h33m35.827308934s) [noise=true , address=104.36.82.46:4490 , piece_id=MFHE7SH22T6KPVQD57THJ47O7ANA3UL6DSXPXE5VKKZGZHCXHWMQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.404358458 (95h33m35.835438599s) [noise=true , address=104.36.82.35:4583 , piece_id=F425SAQ7MSRGMEAFG75XFNNVQPJOXKKXOCF4DXUYB3ZJJX7O3Y2Q, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.424749884 (95h33m35.855830015s) [noise=true , address=104.36.82.16:4297 , piece_id=VDDGO3TNUEPV45LB3J65UZX7WYF4BORGTQ5FZDLOLPUIWGHFDG6Q, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.457148824 (95h33m35.888228945s) [noise=true , address=104.36.82.35:4607 , piece_id=MEGUKOCNGAXTQF6JKS4UM2FTUOJNH22XUUCEOLZG56XXBFRESSRQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.505612983 (95h33m35.936693114s) [noise=true , address=104.36.82.61:4405 , piece_id=42XK2XF4LW6JDERRITW4BGU4SASVH5XAIAI4I7H7WJBMPDD2UVZQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.505632861 (95h33m35.936712992s) [seg_pos={0 0}, upload=99560]: Segment reached optimal threshold of 14 pieces. Aug 12 16:59:41.505886516 (95h33m35.936966647s) [noise=true , address=79.127.201.214:4320 , piece_id=KTDAMSYK7CDMKJDYD7EL45LDWFRECADQYJWINN6BFJUKFCHCXKCQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZLCXYP6sppXZcmj49C8jjgXbLGzgyowtpb1um9XZguzfVnM2r): protocol: expected piece hash; context canceled; EOF Aug 12 16:59:41.505927656 (95h33m35.937007787s) [noise=true , address=104.36.82.23:4467 , piece_id=72OG4FRL66Q522MHHPWUEX5F7TBW7BHTNBUEB2FTLVX6GNEKEBUQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RegmCcGuZED5cPuhVJ9mLRuouCHBEZ3hecf7aseHsMCgo1ajT): protocol: expected piece hash; context canceled; EOF Aug 12 16:59:41.505990275 (95h33m35.937070406s) [noise=true , address=79.127.201.215:4350 , piece_id=J5FYRE2SDV7OCGN2OK3BBY2TZ6JKCJ5TUIPJ67SEMTQRBIJN3E2A, seg_pos={0 0}, upload=99560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WhZAWvHCTj5xVkUuVZtgkFB1dhFKpBPq18YMKkepFx5qxkUAr): protocol: expected piece hash; context canceled; EOF Aug 12 16:59:41.506052915 (95h33m35.937133036s) [noise=true , address=104.36.82.5:4869 , piece_id=TOHQG3BR5YJQNJDMVDWUZ6DLBKNOI5NCOCOI6G2L44BSJ24CR42Q, seg_pos={0 0}, upload=99560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1273igSPBbnYnP5bTMPjCphwkCSK4WdBdBMFVgfnrPaq4oSUZQr): protocol: write tcp 188.166.225.170:39306->104.36.82.5:4869: use of closed network connection; write tcp 188.166.225.170:39306->104.36.82.5:4869: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39306->104.36.82.5:4869: use of closed network connection Aug 12 16:59:41.506075017 (95h33m35.937155148s) [seg_pos={0 0}, upload=99560]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 12 16:59:41.506137476 (95h33m35.937217597s) [upload=99560]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 16:59:41.506141796 (95h33m35.937221927s) [upload=99560]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:59:41.812022822 (95h33m36.243102943s) [upload=99560]: Done uploading segments... Aug 12 16:59:41.812034574 (95h33m36.243114695s) [upload=99560]: Upload finished. err: <nil> Duration: 5.74933419s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............X..........X...................................] duration: 1.851835007s FAIL (2025-08-12 17:06:16 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 12 16:39:09.008146755 (95h13m3.439226887s) [upload=99194]: Starting upload Aug 12 16:39:09.008201260 (95h13m3.439281391s) [upload=99194]: Done starting upload Aug 12 16:39:09.008258329 (95h13m3.43933846s) [upload=99194]: Uploading segments... Aug 12 16:39:09.008290200 (95h13m3.439370331s) [upload=99194]: Got next segment. Inline: true Aug 12 16:39:09.008295661 (95h13m3.439375782s) [upload=99194]: Waiting for error group managing segments... Aug 12 16:39:09.008328123 (95h13m3.439408254s) [upload=99194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:39:09.008334585 (95h13m3.439414716s) [upload=99194]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 16:39:09.008337160 (95h13m3.439417281s) [upload=99194]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:39:09.337170261 (95h13m3.768250392s) [upload=99194]: Done uploading segments... Aug 12 16:39:09.337190520 (95h13m3.768270651s) [upload=99194]: Upload finished. err: <nil> Duration:* 1.85185786s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.815710454s FAIL (2025-08-12 17:06:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 12 16:51:00.342944596 (95h24m54.774024727s) [upload=99416]: Starting upload Aug 12 16:51:00.343003058 (95h24m54.774083189s) [upload=99416]: Done starting upload Aug 12 16:51:00.343093682 (95h24m54.774173803s) [upload=99416]: Uploading segments... Aug 12 16:51:00.343214823 (95h24m54.774294954s) [upload=99416]: Got next segment. Inline: false Aug 12 16:51:00.343269027 (95h24m54.774349158s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:51:00.343273756 (95h24m54.774353887s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 16:51:00.585349151 (95h24m55.016429282s) [seg_pos={0 0}, upload=99416]: Begin upload segment... Aug 12 16:51:00.585396822 (95h24m55.016476973s) [seg_pos={0 0}, upload=99416]: Done begin upload segment. Aug 12 16:51:00.585410067 (95h24m55.016490208s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585507734 (95h24m55.016587895s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.585886159 (95h24m55.0169663s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586089607 (95h24m55.017169738s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.586541652 (95h24m55.017621803s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587004688 (95h24m55.018084819s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587619234 (95h24m55.018699385s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.587983842 (95h24m55.019063983s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588303153 (95h24m55.019383284s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.588688090 (95h24m55.019768221s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.589902393 (95h24m55.020982534s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.590674450 (95h24m55.021754601s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.591296961 (95h24m55.022377092s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.592189969 (95h24m55.02327011s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.593156619 (95h24m55.02423678s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:00.594056290 (95h24m55.025136421s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Uploading piece... Aug 12 16:51:02.362133336 (95h24m56.793213467s) [upload=99416]: Next returned nil segment Aug 12 16:51:02.362159547 (95h24m56.793239678s) [upload=99416]: Waiting for error group managing segments... Aug 12 16:51:02.650293798 (95h24m57.081373929s) [noise=true , address=77.90.31.18:4805 , piece_id=NW27AB2XE452YUWKJOSPZXQQ3O5THGJQZWPFNCYYTTD5KSM4LUIQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.663772758 (95h24m57.094852919s) [noise=true , address=77.90.31.12:4592 , piece_id=D5DIBKFCLXZCN5WGTVASSOPLLGWXBY42VRDRT4GERYQEN7TE7OQQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.696344749 (95h24m57.12742488s) [noise=true , address=77.90.31.17:4772 , piece_id=F335TOCUX6EOQEL3B6CRYVSNT3K3VPQKUIO4OQCKLUOYDPITGCRQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.709104884 (95h24m57.140185025s) [noise=true , address=104.36.82.12:4150 , piece_id=2URQGYBRQFL4GRLQTARGPS5OHFKAIMQ5LK4TBQJKEHWATWJQUIYA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.722233484 (95h24m57.153313625s) [noise=true , address=45.91.116.5:4299 , piece_id=KKS2MUX4QFJSJLZB6FQ7QUSIXNDXG5V666IDIXCZTVLIARPB5OOQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.727048907 (95h24m57.158129038s) [noise=true , address=77.90.31.16:4734 , piece_id=UDY6JVB3TPS2HOLPTX4ILBUJXC6IQC3BOZNVHKANKWELCBZ4O24A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.741870376 (95h24m57.172950507s) [noise=true , address=45.91.116.7:4009 , piece_id=74F32XPA5Z3DQY36ATNSVLFQSIAHA6EHL3T5W2HO2547GFJXWG7A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.772732365 (95h24m57.203812507s) [noise=true , address=104.36.82.14:4222 , piece_id=ZNU7RTT5A4ZIX23ERO2NKXX4QVNKLJB4R3VJRZVQ5SRRI5OWGC2A, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.805755481 (95h24m57.236835612s) [noise=true , address=104.36.82.4:4801 , piece_id=AZNJYD5EDG7BFS35RJP3MLEJCMPM6J7UKQIBA2RIXTXARTJ4FXGQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.878280266 (95h24m57.309360397s) [noise=true , address=77.90.31.10:4520 , piece_id=LQC7NVJGKRVXL34UQO7TDTRE3VAKZ5CENKPKM7JUSW4I4TQ3MZCQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.913734542 (95h24m57.344814673s) [noise=true , address=45.91.116.6:4975 , piece_id=MO7KK43ZBCOIDIQGWFRHNFWG5WZL77ZESIW4U24DXCC5BB57R52Q, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.972369120 (95h24m57.403449251s) [noise=true , address=104.36.82.10:4079 , piece_id=URAOYRC7A52G7O5TOAEPHJHT3G6GBDJZN3GTA37BQ5TBRSW4OYFQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983815812 (95h24m57.414895943s) [noise=true , address=104.36.82.2:4732 , piece_id=N4DXFGAMG72X2ER6AY53WWXX5KKUPAVCOVFWAFQ57KEHHFUXE2GQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: <nil> Aug 12 16:51:02.983842733 (95h24m57.414922864s) [seg_pos={0 0}, upload=99416]: Segment reached optimal threshold of 13 pieces. Aug 12 16:51:02.983942084 (95h24m57.415022215s) [noise=true , address=77.90.31.11:4554 , piece_id=HEXD7AOVOJTK37LL4SCH43UN4VN7EI7G7WFSPOPGHIGBPKTGV3DA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.983999583 (95h24m57.415079715s) [noise=true , address=45.91.116.7:4010 , piece_id=4UT6Y2N3KF4HO6YHIBQVF46Q5JNHLDJYBLLCFXFDOURAH4EVMBKA, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984087211 (95h24m57.415167342s) [noise=true , address=45.91.116.5:4298 , piece_id=FMRPFGWGVYKSC67IBWY5UFDVPN7KKRNOWZ5K66XRL4BSOXM6PVKQ, seg_pos={0 0}, upload=99416]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Aug 12 16:51:02.984124363 (95h24m57.415204494s) [seg_pos={0 0}, upload=99416]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 12 16:51:02.984229683 (95h24m57.415309814s) [upload=99416]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 16:51:02.984234323 (95h24m57.415314454s) [upload=99416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:51:03.289516404 (95h24m57.720596575s) [upload=99416]: Done uploading segments... Aug 12 16:51:03.289540049 (95h24m57.7206202s) [upload=99416]: Upload finished. err: <nil> Duration: 5.815732735s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....X.X...X.............................X.X.........X......] duration: 6.060190963s FAIL (2025-08-12 17:05:56 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 12 16:59:38.687530140 (95h33m33.118610271s) [upload=99560]: Starting upload Aug 12 16:59:38.687574635 (95h33m33.118654766s) [upload=99560]: Done starting upload Aug 12 16:59:38.687634360 (95h33m33.118714491s) [upload=99560]: Uploading segments... Aug 12 16:59:38.687738799 (95h33m33.11881892s) [upload=99560]: Got next segment. Inline: false Aug 12 16:59:38.687772675 (95h33m33.118852806s) [upload=99560]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:59:38.687777594 (95h33m33.118857725s) [upload=99560]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 12 16:59:39.197902742 (95h33m33.628982873s) [seg_pos={0 0}, upload=99560]: Begin upload segment... Aug 12 16:59:39.197949550 (95h33m33.629029682s) [seg_pos={0 0}, upload=99560]: Done begin upload segment. Aug 12 16:59:39.197963598 (95h33m33.629043729s) [noise=true , address=104.36.82.21:4495 , piece_id=6PW2LL4YAWQO6EGJWRPGO7PDK7VRJ63Y6E5FIS3FIMQDQS37JE2Q, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.197979778 (95h33m33.629059909s) [noise=true , address=104.36.82.61:4405 , piece_id=42XK2XF4LW6JDERRITW4BGU4SASVH5XAIAI4I7H7WJBMPDD2UVZQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198019174 (95h33m33.629099305s) [noise=true , address=104.36.82.62:4410 , piece_id=NNHLCIVHJSFAQOF5BPARODUZ7NDKRIO7VZ4TXMDPSIT7CPBM7LGA, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198062236 (95h33m33.629142357s) [noise=true , address=77.90.31.19:4844 , piece_id=6ACO5XJCM7JKXI6DZNS7Q6FMNIIF2H4QX5GFFZP275LGK4LAG6KQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198334707 (95h33m33.629414828s) [noise=true , address=104.36.82.58:4285 , piece_id=RSNIOADYVBK6XD3XNUUNRLYBSDKJNLXTGGQSUQA5MF2KOGCY7WVQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198339377 (95h33m33.629419508s) [noise=true , address=79.127.201.214:4320 , piece_id=KTDAMSYK7CDMKJDYD7EL45LDWFRECADQYJWINN6BFJUKFCHCXKCQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198607269 (95h33m33.6296874s) [noise=true , address=104.36.82.23:4467 , piece_id=72OG4FRL66Q522MHHPWUEX5F7TBW7BHTNBUEB2FTLVX6GNEKEBUQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.198865934 (95h33m33.629946065s) [noise=true , address=104.36.82.35:4607 , piece_id=MEGUKOCNGAXTQF6JKS4UM2FTUOJNH22XUUCEOLZG56XXBFRESSRQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.200169648 (95h33m33.631249779s) [noise=true , address=104.36.82.35:4583 , piece_id=F425SAQ7MSRGMEAFG75XFNNVQPJOXKKXOCF4DXUYB3ZJJX7O3Y2Q, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.200909183 (95h33m33.631989315s) [noise=true , address=104.36.82.3:4796 , piece_id=GKGL3BVJKJI34563P6NAAMQBH6K53WQ2XR5A5TRPTA4VK24WN7HA, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.201550832 (95h33m33.632630963s) [noise=true , address=104.36.82.47:4519 , piece_id=CWQ2QS6LJLG2MGBY3B5KHVUKDJQ64ZAWHCQSEIMBHYXVZ2KQONWA, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.202192539 (95h33m33.63327267s) [noise=true , address=193.149.164.143:4698 , piece_id=I7BDBG7QK66PCZNNU3MDNT3IENTGDLSMHIQMFDAJGGGLTWNNW7IA, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.202627732 (95h33m33.633707863s) [noise=true , address=104.36.82.5:4869 , piece_id=TOHQG3BR5YJQNJDMVDWUZ6DLBKNOI5NCOCOI6G2L44BSJ24CR42Q, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.202954337 (95h33m33.634034468s) [noise=true , address=104.36.82.46:4490 , piece_id=MFHE7SH22T6KPVQD57THJ47O7ANA3UL6DSXPXE5VKKZGZHCXHWMQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.203178796 (95h33m33.634258927s) [noise=true , address=104.36.82.18:4366 , piece_id=ABVSGRVBMLWFCYO4RGTYAZ4XMYFCKKL4OMFAMS2WLD4OFNCHKUZQ, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.203380622 (95h33m33.634460753s) [noise=true , address=104.36.82.16:4297 , piece_id=VDDGO3TNUEPV45LB3J65UZX7WYF4BORGTQ5FZDLOLPUIWGHFDG6Q, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.203821095 (95h33m33.634901226s) [noise=true , address=79.127.201.215:4350 , piece_id=J5FYRE2SDV7OCGN2OK3BBY2TZ6JKCJ5TUIPJ67SEMTQRBIJN3E2A, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:39.205764854 (95h33m33.636844995s) [noise=true , address=104.36.82.60:4372 , piece_id=M54ATKGBFCQWGXWTCFU7GDGGWPTN5UJONPFDT6QEW4BP5CJCRKIA, seg_pos={0 0}, upload=99560]: Uploading piece... Aug 12 16:59:40.363639120 (95h33m34.794719261s) [upload=99560]: Next returned nil segment Aug 12 16:59:40.363657104 (95h33m34.794737235s) [upload=99560]: Waiting for error group managing segments... Aug 12 16:59:40.692247382 (95h33m35.123327513s) [noise=true , address=193.149.164.143:4698 , piece_id=I7BDBG7QK66PCZNNU3MDNT3IENTGDLSMHIQMFDAJGGGLTWNNW7IA, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.169955785 (95h33m35.601035916s) [noise=true , address=104.36.82.21:4495 , piece_id=6PW2LL4YAWQO6EGJWRPGO7PDK7VRJ63Y6E5FIS3FIMQDQS37JE2Q, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.173959115 (95h33m35.605039236s) [noise=true , address=77.90.31.19:4844 , piece_id=6ACO5XJCM7JKXI6DZNS7Q6FMNIIF2H4QX5GFFZP275LGK4LAG6KQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.202640181 (95h33m35.633720302s) [noise=true , address=104.36.82.62:4410 , piece_id=NNHLCIVHJSFAQOF5BPARODUZ7NDKRIO7VZ4TXMDPSIT7CPBM7LGA, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.217521444 (95h33m35.648601575s) [noise=true , address=104.36.82.47:4519 , piece_id=CWQ2QS6LJLG2MGBY3B5KHVUKDJQ64ZAWHCQSEIMBHYXVZ2KQONWA, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.246049799 (95h33m35.67712993s) [noise=true , address=104.36.82.3:4796 , piece_id=GKGL3BVJKJI34563P6NAAMQBH6K53WQ2XR5A5TRPTA4VK24WN7HA, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.264431349 (95h33m35.69551148s) [noise=true , address=104.36.82.18:4366 , piece_id=ABVSGRVBMLWFCYO4RGTYAZ4XMYFCKKL4OMFAMS2WLD4OFNCHKUZQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.367834365 (95h33m35.798914496s) [noise=true , address=104.36.82.60:4372 , piece_id=M54ATKGBFCQWGXWTCFU7GDGGWPTN5UJONPFDT6QEW4BP5CJCRKIA, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.368115543 (95h33m35.799195664s) [noise=true , address=104.36.82.58:4285 , piece_id=RSNIOADYVBK6XD3XNUUNRLYBSDKJNLXTGGQSUQA5MF2KOGCY7WVQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.396228793 (95h33m35.827308934s) [noise=true , address=104.36.82.46:4490 , piece_id=MFHE7SH22T6KPVQD57THJ47O7ANA3UL6DSXPXE5VKKZGZHCXHWMQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.404358458 (95h33m35.835438599s) [noise=true , address=104.36.82.35:4583 , piece_id=F425SAQ7MSRGMEAFG75XFNNVQPJOXKKXOCF4DXUYB3ZJJX7O3Y2Q, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.424749884 (95h33m35.855830015s) [noise=true , address=104.36.82.16:4297 , piece_id=VDDGO3TNUEPV45LB3J65UZX7WYF4BORGTQ5FZDLOLPUIWGHFDG6Q, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.457148824 (95h33m35.888228945s) [noise=true , address=104.36.82.35:4607 , piece_id=MEGUKOCNGAXTQF6JKS4UM2FTUOJNH22XUUCEOLZG56XXBFRESSRQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.505612983 (95h33m35.936693114s) [noise=true , address=104.36.82.61:4405 , piece_id=42XK2XF4LW6JDERRITW4BGU4SASVH5XAIAI4I7H7WJBMPDD2UVZQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: <nil> Aug 12 16:59:41.505632861 (95h33m35.936712992s) [seg_pos={0 0}, upload=99560]: Segment reached optimal threshold of 14 pieces. Aug 12 16:59:41.505886516 (95h33m35.936966647s) [noise=true , address=79.127.201.214:4320 , piece_id=KTDAMSYK7CDMKJDYD7EL45LDWFRECADQYJWINN6BFJUKFCHCXKCQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZLCXYP6sppXZcmj49C8jjgXbLGzgyowtpb1um9XZguzfVnM2r): protocol: expected piece hash; context canceled; EOF Aug 12 16:59:41.505927656 (95h33m35.937007787s) [noise=true , address=104.36.82.23:4467 , piece_id=72OG4FRL66Q522MHHPWUEX5F7TBW7BHTNBUEB2FTLVX6GNEKEBUQ, seg_pos={0 0}, upload=99560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RegmCcGuZED5cPuhVJ9mLRuouCHBEZ3hecf7aseHsMCgo1ajT): protocol: expected piece hash; context canceled; EOF Aug 12 16:59:41.505990275 (95h33m35.937070406s) [noise=true , address=79.127.201.215:4350 , piece_id=J5FYRE2SDV7OCGN2OK3BBY2TZ6JKCJ5TUIPJ67SEMTQRBIJN3E2A, seg_pos={0 0}, upload=99560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WhZAWvHCTj5xVkUuVZtgkFB1dhFKpBPq18YMKkepFx5qxkUAr): protocol: expected piece hash; context canceled; EOF Aug 12 16:59:41.506052915 (95h33m35.937133036s) [noise=true , address=104.36.82.5:4869 , piece_id=TOHQG3BR5YJQNJDMVDWUZ6DLBKNOI5NCOCOI6G2L44BSJ24CR42Q, seg_pos={0 0}, upload=99560]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1273igSPBbnYnP5bTMPjCphwkCSK4WdBdBMFVgfnrPaq4oSUZQr): protocol: write tcp 188.166.225.170:39306->104.36.82.5:4869: use of closed network connection; write tcp 188.166.225.170:39306->104.36.82.5:4869: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39306->104.36.82.5:4869: use of closed network connection Aug 12 16:59:41.506075017 (95h33m35.937155148s) [seg_pos={0 0}, upload=99560]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 12 16:59:41.506137476 (95h33m35.937217597s) [upload=99560]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 12 16:59:41.506141796 (95h33m35.937221927s) [upload=99560]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:59:41.812022822 (95h33m36.243102943s) [upload=99560]: Done uploading segments... Aug 12 16:59:41.812034574 (95h33m36.243114695s) [upload=99560]: Upload finished. err: <nil> Duration: 6.060222724s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............X..........X..................................] duration: 1.872529453s FAIL (2025-08-12 17:05:45 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 12 16:39:09.008146755 (95h13m3.439226887s) [upload=99194]: Starting upload Aug 12 16:39:09.008201260 (95h13m3.439281391s) [upload=99194]: Done starting upload Aug 12 16:39:09.008258329 (95h13m3.43933846s) [upload=99194]: Uploading segments... Aug 12 16:39:09.008290200 (95h13m3.439370331s) [upload=99194]: Got next segment. Inline: true Aug 12 16:39:09.008295661 (95h13m3.439375782s) [upload=99194]: Waiting for error group managing segments... Aug 12 16:39:09.008328123 (95h13m3.439408254s) [upload=99194]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 12 16:39:09.008334585 (95h13m3.439414716s) [upload=99194]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 12 16:39:09.008337160 (95h13m3.439417281s) [upload=99194]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 12 16:39:09.337170261 (95h13m3.768250392s) [upload=99194]: Done uploading segments... Aug 12 16:39:09.337190520 (95h13m3.768270651s) [upload=99194]: Upload finished. err: <nil> Duration:* 1.87255976s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)