Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............X...................................X..........] duration: 5.78512595s FAIL (2025-08-21 20:16:37 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 21 20:10:14.716582768 (314h44m9.147662899s) [upload=327443]: Starting upload Aug 21 20:10:14.716649245 (314h44m9.147729366s) [upload=327443]: Done starting upload Aug 21 20:10:14.716772782 (314h44m9.147852913s) [upload=327443]: Uploading segments... Aug 21 20:10:14.716914553 (314h44m9.147994684s) [upload=327443]: Got next segment. Inline: false Aug 21 20:10:14.716961734 (314h44m9.148041855s) [upload=327443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:10:14.716966182 (314h44m9.148046313s) [upload=327443]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 21 20:10:15.167114827 (314h44m9.598194958s) [seg_pos={0 0}, upload=327443]: Begin upload segment... Aug 21 20:10:15.167157349 (314h44m9.59823748s) [seg_pos={0 0}, upload=327443]: Done begin upload segment. Aug 21 20:10:15.167177778 (314h44m9.598257899s) [noise=true , address=77.90.31.17:4785 , piece_id=XKOCSZKVZ63UDGWWXP6DO6FVAWHNUSZLQZ75AYL72JSFA3LND7PQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167269874 (314h44m9.598349995s) [noise=true , address=45.91.116.4:4962 , piece_id=7W2H6URV32LWUBDIDSMFL4HS6DRZ5GFQAU6G25Y3PNJHYDIVCPZA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167549559 (314h44m9.59862969s) [noise=true , address=104.36.82.11:4126 , piece_id=246UC4PSMOZ2Q7557TLR4QUON7X2DW7SYQLH7534XSEDRKFK6YTA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167820397 (314h44m9.598900528s) [noise=true , address=77.90.31.17:4775 , piece_id=ZGZNQUHXPKQMWYHUPES6JLK74YGWSOWSRVPTOCKEUQQGWNQEG2DQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167897014 (314h44m9.598977135s) [noise=true , address=104.36.82.38:4713 , piece_id=RWBXRNQ2JMFJ4SYHSVGRJPUYUQMQN2URNCN6LJFW3PDVS6EVDY4Q, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168095995 (314h44m9.599176126s) [noise=true , address=104.36.82.5:4843 , piece_id=Z7WZ5UTYX32KJADY3FSEJGPAKCNG7LU5FEPXZ4TAXXOGFQ4EYBEA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168341745 (314h44m9.599421866s) [noise=true , address=45.91.116.12:4255 , piece_id=TNSZHSIHY7JDMMW6D3TL42AFSE2OPVVAK2NMK6C52VBWMHUHPQ7A, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168407420 (314h44m9.599487551s) [noise=true , address=79.127.201.215:4344 , piece_id=2JUCTHQ3BI3B3SE3AQFC63L3GNLYMYHTWLRRHLDXC3CCTHS5EGKA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168667458 (314h44m9.599747579s) [noise=true , address=45.91.116.10:4114 , piece_id=REEPZ6DH3SKWEHY2N67ZCVVCOPSEZEFT366BC4JVBI7GYJ4WKIGQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168907448 (314h44m9.599987569s) [noise=true , address=77.90.31.12:4594 , piece_id=WDHXZIEPQRHWZHTWJ6CGMIRNIQAI3NZ7YVP2IICUGBWS3SUB77FA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168965298 (314h44m9.600045429s) [noise=true , address=77.90.31.11:4559 , piece_id=6NWPHRGM3VPBZG7IV5755VGSMSG5KD2S6FQYS7RJZ63N52L7UBDQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169160421 (314h44m9.600240553s) [noise=true , address=104.36.82.34:4570 , piece_id=SG6AVXZ6JMY6BZMPO7YO4D2M4GSKMHFYQJKBHQFNXB2OWXX7PKXA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169288357 (314h44m9.600368488s) [noise=true , address=104.36.82.13:4212 , piece_id=YYDY7XTFA6EGFPWTPSFRIDSRJYTJLTKM54UIDQUYN5ZM7OVPCK5Q, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169361095 (314h44m9.600441216s) [noise=true , address=104.36.82.23:4477 , piece_id=LQWY3THAYVTLQQWDKSM445MVXVDF2DQCEDLBROM7DONFMXSSSAVA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169555206 (314h44m9.600635337s) [noise=true , address=45.91.116.7:4014 , piece_id=SPFLBQZCPK33TXQ6IBHFCB3WH7XO4PHREI5MWVPNCEQF4EYZP5RQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169563844 (314h44m9.600643975s) [noise=true , address=104.36.82.40:4963 , piece_id=6VIZW4MFPEOVQORZYUS2VOAA6Q4U66ZWZEHFXKILJTSRF2RQLNSA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169877724 (314h44m9.600957845s) [noise=true , address=104.36.82.9:4018 , piece_id=7S4G6XMWKXQ5U3FQDMW2VDLZYWFOBVRKMCFED6BCCIWE4QAA7I2A, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.170176706 (314h44m9.601256837s) [noise=true , address=104.36.82.2:4762 , piece_id=OIUTGZSMAVR2HYTRWWMWLSR5XTXJQHFXWA6D2WU5JDNBXLRYWCOA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:16.672535880 (314h44m11.103616011s) [upload=327443]: Next returned nil segment Aug 21 20:10:16.672555797 (314h44m11.103635928s) [upload=327443]: Waiting for error group managing segments... Aug 21 20:10:17.083344442 (314h44m11.514424563s) [noise=true , address=45.91.116.7:4014 , piece_id=SPFLBQZCPK33TXQ6IBHFCB3WH7XO4PHREI5MWVPNCEQF4EYZP5RQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.083887000 (314h44m11.514967132s) [noise=true , address=45.91.116.4:4962 , piece_id=7W2H6URV32LWUBDIDSMFL4HS6DRZ5GFQAU6G25Y3PNJHYDIVCPZA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.090288509 (314h44m11.52136868s) [noise=true , address=77.90.31.17:4785 , piece_id=XKOCSZKVZ63UDGWWXP6DO6FVAWHNUSZLQZ75AYL72JSFA3LND7PQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.141165807 (314h44m11.572245938s) [noise=true , address=104.36.82.9:4018 , piece_id=7S4G6XMWKXQ5U3FQDMW2VDLZYWFOBVRKMCFED6BCCIWE4QAA7I2A, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.157740550 (314h44m11.588820682s) [noise=true , address=45.91.116.12:4255 , piece_id=TNSZHSIHY7JDMMW6D3TL42AFSE2OPVVAK2NMK6C52VBWMHUHPQ7A, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.158401245 (314h44m11.589481376s) [noise=true , address=77.90.31.11:4559 , piece_id=6NWPHRGM3VPBZG7IV5755VGSMSG5KD2S6FQYS7RJZ63N52L7UBDQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.170813964 (314h44m11.601894085s) [noise=true , address=45.91.116.10:4114 , piece_id=REEPZ6DH3SKWEHY2N67ZCVVCOPSEZEFT366BC4JVBI7GYJ4WKIGQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.175584983 (314h44m11.606665114s) [noise=true , address=79.127.201.215:4344 , piece_id=2JUCTHQ3BI3B3SE3AQFC63L3GNLYMYHTWLRRHLDXC3CCTHS5EGKA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.197949946 (314h44m11.629030077s) [noise=true , address=77.90.31.12:4594 , piece_id=WDHXZIEPQRHWZHTWJ6CGMIRNIQAI3NZ7YVP2IICUGBWS3SUB77FA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.202226388 (314h44m11.633306509s) [noise=true , address=104.36.82.2:4762 , piece_id=OIUTGZSMAVR2HYTRWWMWLSR5XTXJQHFXWA6D2WU5JDNBXLRYWCOA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.202498088 (314h44m11.633578209s) [noise=true , address=104.36.82.11:4126 , piece_id=246UC4PSMOZ2Q7557TLR4QUON7X2DW7SYQLH7534XSEDRKFK6YTA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.204637111 (314h44m11.635717232s) [noise=true , address=104.36.82.34:4570 , piece_id=SG6AVXZ6JMY6BZMPO7YO4D2M4GSKMHFYQJKBHQFNXB2OWXX7PKXA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.208072624 (314h44m11.639152755s) [noise=true , address=104.36.82.40:4963 , piece_id=6VIZW4MFPEOVQORZYUS2VOAA6Q4U66ZWZEHFXKILJTSRF2RQLNSA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.251397182 (314h44m11.682477313s) [noise=true , address=104.36.82.38:4713 , piece_id=RWBXRNQ2JMFJ4SYHSVGRJPUYUQMQN2URNCN6LJFW3PDVS6EVDY4Q, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.251419764 (314h44m11.682499895s) [seg_pos={0 0}, upload=327443]: Segment reached optimal threshold of 14 pieces. Aug 21 20:10:17.251567658 (314h44m11.682647789s) [noise=true , address=77.90.31.17:4775 , piece_id=ZGZNQUHXPKQMWYHUPES6JLK74YGWSOWSRVPTOCKEUQQGWNQEG2DQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127VKvwi82kYfGZ6Wcv8BinaNUQBMQsgKLKNNLht5JPsBLN7ovt): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251639294 (314h44m11.682719425s) [noise=true , address=104.36.82.23:4477 , piece_id=LQWY3THAYVTLQQWDKSM445MVXVDF2DQCEDLBROM7DONFMXSSSAVA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RP2Zq643WWNtKEr34Tq1iZS29NYA2okr9Fg1jiCpaFTqfLauf): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251720610 (314h44m11.682800741s) [noise=true , address=104.36.82.5:4843 , piece_id=Z7WZ5UTYX32KJADY3FSEJGPAKCNG7LU5FEPXZ4TAXXOGFQ4EYBEA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HvL2JDP4GCsu3tziVEKRMh14nEzyRMRuWNSmVvs27LzHTbNpS): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251764264 (314h44m11.682844395s) [noise=true , address=104.36.82.13:4212 , piece_id=YYDY7XTFA6EGFPWTPSFRIDSRJYTJLTKM54UIDQUYN5ZM7OVPCK5Q, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1itAUtWeFERY4AKhPxE2CRwymp8qNC6wu6uhPt2n6aK58LRu24): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251780665 (314h44m11.682860796s) [seg_pos={0 0}, upload=327443]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 21 20:10:17.251866870 (314h44m11.682947001s) [upload=327443]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 21 20:10:17.251871469 (314h44m11.6829516s) [upload=327443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:10:19.201398003 (314h44m13.632478134s) [upload=327443]: Done uploading segments... Aug 21 20:10:19.201416889 (314h44m13.63249702s) [upload=327443]: Upload finished. err: <nil> Duration: 5.785157649s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..........X..............................X................X.] duration: 2.006087337s FAIL (2025-08-21 20:16:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== Aug 21 20:15:43.345939762 (314h49m37.777019883s) [upload=327543]: Starting upload Aug 21 20:15:43.345985320 (314h49m37.777065451s) [upload=327543]: Done starting upload Aug 21 20:15:43.346041136 (314h49m37.777121257s) [upload=327543]: Uploading segments... Aug 21 20:15:43.346076434 (314h49m37.777156565s) [upload=327543]: Got next segment. Inline: true Aug 21 20:15:43.346080321 (314h49m37.777160452s) [upload=327543]: Waiting for error group managing segments... Aug 21 20:15:43.346111561 (314h49m37.777191682s) [upload=327543]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:15:43.346116079 (314h49m37.7771962s) [upload=327543]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 21 20:15:43.346118665 (314h49m37.777198796s) [upload=327543]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:15:43.644540599 (314h49m38.07562072s) [upload=327543]: Done uploading segments... Aug 21 20:15:43.644561158 (314h49m38.075641279s) [upload=327543]: Upload finished. err: <nil> Duration:* 2.006106826s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............X...................................X.........] duration: 5.978058888s FAIL (2025-08-21 20:16:09 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 21 20:10:14.716582768 (314h44m9.147662899s) [upload=327443]: Starting upload Aug 21 20:10:14.716649245 (314h44m9.147729366s) [upload=327443]: Done starting upload Aug 21 20:10:14.716772782 (314h44m9.147852913s) [upload=327443]: Uploading segments... Aug 21 20:10:14.716914553 (314h44m9.147994684s) [upload=327443]: Got next segment. Inline: false Aug 21 20:10:14.716961734 (314h44m9.148041855s) [upload=327443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:10:14.716966182 (314h44m9.148046313s) [upload=327443]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 21 20:10:15.167114827 (314h44m9.598194958s) [seg_pos={0 0}, upload=327443]: Begin upload segment... Aug 21 20:10:15.167157349 (314h44m9.59823748s) [seg_pos={0 0}, upload=327443]: Done begin upload segment. Aug 21 20:10:15.167177778 (314h44m9.598257899s) [noise=true , address=77.90.31.17:4785 , piece_id=XKOCSZKVZ63UDGWWXP6DO6FVAWHNUSZLQZ75AYL72JSFA3LND7PQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167269874 (314h44m9.598349995s) [noise=true , address=45.91.116.4:4962 , piece_id=7W2H6URV32LWUBDIDSMFL4HS6DRZ5GFQAU6G25Y3PNJHYDIVCPZA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167549559 (314h44m9.59862969s) [noise=true , address=104.36.82.11:4126 , piece_id=246UC4PSMOZ2Q7557TLR4QUON7X2DW7SYQLH7534XSEDRKFK6YTA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167820397 (314h44m9.598900528s) [noise=true , address=77.90.31.17:4775 , piece_id=ZGZNQUHXPKQMWYHUPES6JLK74YGWSOWSRVPTOCKEUQQGWNQEG2DQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167897014 (314h44m9.598977135s) [noise=true , address=104.36.82.38:4713 , piece_id=RWBXRNQ2JMFJ4SYHSVGRJPUYUQMQN2URNCN6LJFW3PDVS6EVDY4Q, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168095995 (314h44m9.599176126s) [noise=true , address=104.36.82.5:4843 , piece_id=Z7WZ5UTYX32KJADY3FSEJGPAKCNG7LU5FEPXZ4TAXXOGFQ4EYBEA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168341745 (314h44m9.599421866s) [noise=true , address=45.91.116.12:4255 , piece_id=TNSZHSIHY7JDMMW6D3TL42AFSE2OPVVAK2NMK6C52VBWMHUHPQ7A, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168407420 (314h44m9.599487551s) [noise=true , address=79.127.201.215:4344 , piece_id=2JUCTHQ3BI3B3SE3AQFC63L3GNLYMYHTWLRRHLDXC3CCTHS5EGKA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168667458 (314h44m9.599747579s) [noise=true , address=45.91.116.10:4114 , piece_id=REEPZ6DH3SKWEHY2N67ZCVVCOPSEZEFT366BC4JVBI7GYJ4WKIGQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168907448 (314h44m9.599987569s) [noise=true , address=77.90.31.12:4594 , piece_id=WDHXZIEPQRHWZHTWJ6CGMIRNIQAI3NZ7YVP2IICUGBWS3SUB77FA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168965298 (314h44m9.600045429s) [noise=true , address=77.90.31.11:4559 , piece_id=6NWPHRGM3VPBZG7IV5755VGSMSG5KD2S6FQYS7RJZ63N52L7UBDQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169160421 (314h44m9.600240553s) [noise=true , address=104.36.82.34:4570 , piece_id=SG6AVXZ6JMY6BZMPO7YO4D2M4GSKMHFYQJKBHQFNXB2OWXX7PKXA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169288357 (314h44m9.600368488s) [noise=true , address=104.36.82.13:4212 , piece_id=YYDY7XTFA6EGFPWTPSFRIDSRJYTJLTKM54UIDQUYN5ZM7OVPCK5Q, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169361095 (314h44m9.600441216s) [noise=true , address=104.36.82.23:4477 , piece_id=LQWY3THAYVTLQQWDKSM445MVXVDF2DQCEDLBROM7DONFMXSSSAVA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169555206 (314h44m9.600635337s) [noise=true , address=45.91.116.7:4014 , piece_id=SPFLBQZCPK33TXQ6IBHFCB3WH7XO4PHREI5MWVPNCEQF4EYZP5RQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169563844 (314h44m9.600643975s) [noise=true , address=104.36.82.40:4963 , piece_id=6VIZW4MFPEOVQORZYUS2VOAA6Q4U66ZWZEHFXKILJTSRF2RQLNSA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169877724 (314h44m9.600957845s) [noise=true , address=104.36.82.9:4018 , piece_id=7S4G6XMWKXQ5U3FQDMW2VDLZYWFOBVRKMCFED6BCCIWE4QAA7I2A, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.170176706 (314h44m9.601256837s) [noise=true , address=104.36.82.2:4762 , piece_id=OIUTGZSMAVR2HYTRWWMWLSR5XTXJQHFXWA6D2WU5JDNBXLRYWCOA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:16.672535880 (314h44m11.103616011s) [upload=327443]: Next returned nil segment Aug 21 20:10:16.672555797 (314h44m11.103635928s) [upload=327443]: Waiting for error group managing segments... Aug 21 20:10:17.083344442 (314h44m11.514424563s) [noise=true , address=45.91.116.7:4014 , piece_id=SPFLBQZCPK33TXQ6IBHFCB3WH7XO4PHREI5MWVPNCEQF4EYZP5RQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.083887000 (314h44m11.514967132s) [noise=true , address=45.91.116.4:4962 , piece_id=7W2H6URV32LWUBDIDSMFL4HS6DRZ5GFQAU6G25Y3PNJHYDIVCPZA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.090288509 (314h44m11.52136868s) [noise=true , address=77.90.31.17:4785 , piece_id=XKOCSZKVZ63UDGWWXP6DO6FVAWHNUSZLQZ75AYL72JSFA3LND7PQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.141165807 (314h44m11.572245938s) [noise=true , address=104.36.82.9:4018 , piece_id=7S4G6XMWKXQ5U3FQDMW2VDLZYWFOBVRKMCFED6BCCIWE4QAA7I2A, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.157740550 (314h44m11.588820682s) [noise=true , address=45.91.116.12:4255 , piece_id=TNSZHSIHY7JDMMW6D3TL42AFSE2OPVVAK2NMK6C52VBWMHUHPQ7A, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.158401245 (314h44m11.589481376s) [noise=true , address=77.90.31.11:4559 , piece_id=6NWPHRGM3VPBZG7IV5755VGSMSG5KD2S6FQYS7RJZ63N52L7UBDQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.170813964 (314h44m11.601894085s) [noise=true , address=45.91.116.10:4114 , piece_id=REEPZ6DH3SKWEHY2N67ZCVVCOPSEZEFT366BC4JVBI7GYJ4WKIGQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.175584983 (314h44m11.606665114s) [noise=true , address=79.127.201.215:4344 , piece_id=2JUCTHQ3BI3B3SE3AQFC63L3GNLYMYHTWLRRHLDXC3CCTHS5EGKA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.197949946 (314h44m11.629030077s) [noise=true , address=77.90.31.12:4594 , piece_id=WDHXZIEPQRHWZHTWJ6CGMIRNIQAI3NZ7YVP2IICUGBWS3SUB77FA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.202226388 (314h44m11.633306509s) [noise=true , address=104.36.82.2:4762 , piece_id=OIUTGZSMAVR2HYTRWWMWLSR5XTXJQHFXWA6D2WU5JDNBXLRYWCOA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.202498088 (314h44m11.633578209s) [noise=true , address=104.36.82.11:4126 , piece_id=246UC4PSMOZ2Q7557TLR4QUON7X2DW7SYQLH7534XSEDRKFK6YTA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.204637111 (314h44m11.635717232s) [noise=true , address=104.36.82.34:4570 , piece_id=SG6AVXZ6JMY6BZMPO7YO4D2M4GSKMHFYQJKBHQFNXB2OWXX7PKXA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.208072624 (314h44m11.639152755s) [noise=true , address=104.36.82.40:4963 , piece_id=6VIZW4MFPEOVQORZYUS2VOAA6Q4U66ZWZEHFXKILJTSRF2RQLNSA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.251397182 (314h44m11.682477313s) [noise=true , address=104.36.82.38:4713 , piece_id=RWBXRNQ2JMFJ4SYHSVGRJPUYUQMQN2URNCN6LJFW3PDVS6EVDY4Q, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.251419764 (314h44m11.682499895s) [seg_pos={0 0}, upload=327443]: Segment reached optimal threshold of 14 pieces. Aug 21 20:10:17.251567658 (314h44m11.682647789s) [noise=true , address=77.90.31.17:4775 , piece_id=ZGZNQUHXPKQMWYHUPES6JLK74YGWSOWSRVPTOCKEUQQGWNQEG2DQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127VKvwi82kYfGZ6Wcv8BinaNUQBMQsgKLKNNLht5JPsBLN7ovt): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251639294 (314h44m11.682719425s) [noise=true , address=104.36.82.23:4477 , piece_id=LQWY3THAYVTLQQWDKSM445MVXVDF2DQCEDLBROM7DONFMXSSSAVA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RP2Zq643WWNtKEr34Tq1iZS29NYA2okr9Fg1jiCpaFTqfLauf): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251720610 (314h44m11.682800741s) [noise=true , address=104.36.82.5:4843 , piece_id=Z7WZ5UTYX32KJADY3FSEJGPAKCNG7LU5FEPXZ4TAXXOGFQ4EYBEA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HvL2JDP4GCsu3tziVEKRMh14nEzyRMRuWNSmVvs27LzHTbNpS): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251764264 (314h44m11.682844395s) [noise=true , address=104.36.82.13:4212 , piece_id=YYDY7XTFA6EGFPWTPSFRIDSRJYTJLTKM54UIDQUYN5ZM7OVPCK5Q, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1itAUtWeFERY4AKhPxE2CRwymp8qNC6wu6uhPt2n6aK58LRu24): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251780665 (314h44m11.682860796s) [seg_pos={0 0}, upload=327443]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 21 20:10:17.251866870 (314h44m11.682947001s) [upload=327443]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 21 20:10:17.251871469 (314h44m11.6829516s) [upload=327443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:10:19.201398003 (314h44m13.632478134s) [upload=327443]: Done uploading segments... Aug 21 20:10:19.201416889 (314h44m13.63249702s) [upload=327443]: Upload finished. err: <nil> Duration: 5.978090409s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...........X..............................X................X] duration: 1.643052062s FAIL (2025-08-21 20:15:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== Aug 21 20:15:43.345939762 (314h49m37.777019883s) [upload=327543]: Starting upload Aug 21 20:15:43.345985320 (314h49m37.777065451s) [upload=327543]: Done starting upload Aug 21 20:15:43.346041136 (314h49m37.777121257s) [upload=327543]: Uploading segments... Aug 21 20:15:43.346076434 (314h49m37.777156565s) [upload=327543]: Got next segment. Inline: true Aug 21 20:15:43.346080321 (314h49m37.777160452s) [upload=327543]: Waiting for error group managing segments... Aug 21 20:15:43.346111561 (314h49m37.777191682s) [upload=327543]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:15:43.346116079 (314h49m37.7771962s) [upload=327543]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 21 20:15:43.346118665 (314h49m37.777198796s) [upload=327543]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:15:43.644540599 (314h49m38.07562072s) [upload=327543]: Done uploading segments... Aug 21 20:15:43.644561158 (314h49m38.075641279s) [upload=327543]: Upload finished. err: <nil> Duration:* 1.643089101s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............X...................................X........] duration: 5.79416149s FAIL (2025-08-21 20:15:31 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 21 20:10:14.716582768 (314h44m9.147662899s) [upload=327443]: Starting upload Aug 21 20:10:14.716649245 (314h44m9.147729366s) [upload=327443]: Done starting upload Aug 21 20:10:14.716772782 (314h44m9.147852913s) [upload=327443]: Uploading segments... Aug 21 20:10:14.716914553 (314h44m9.147994684s) [upload=327443]: Got next segment. Inline: false Aug 21 20:10:14.716961734 (314h44m9.148041855s) [upload=327443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:10:14.716966182 (314h44m9.148046313s) [upload=327443]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 21 20:10:15.167114827 (314h44m9.598194958s) [seg_pos={0 0}, upload=327443]: Begin upload segment... Aug 21 20:10:15.167157349 (314h44m9.59823748s) [seg_pos={0 0}, upload=327443]: Done begin upload segment. Aug 21 20:10:15.167177778 (314h44m9.598257899s) [noise=true , address=77.90.31.17:4785 , piece_id=XKOCSZKVZ63UDGWWXP6DO6FVAWHNUSZLQZ75AYL72JSFA3LND7PQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167269874 (314h44m9.598349995s) [noise=true , address=45.91.116.4:4962 , piece_id=7W2H6URV32LWUBDIDSMFL4HS6DRZ5GFQAU6G25Y3PNJHYDIVCPZA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167549559 (314h44m9.59862969s) [noise=true , address=104.36.82.11:4126 , piece_id=246UC4PSMOZ2Q7557TLR4QUON7X2DW7SYQLH7534XSEDRKFK6YTA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167820397 (314h44m9.598900528s) [noise=true , address=77.90.31.17:4775 , piece_id=ZGZNQUHXPKQMWYHUPES6JLK74YGWSOWSRVPTOCKEUQQGWNQEG2DQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167897014 (314h44m9.598977135s) [noise=true , address=104.36.82.38:4713 , piece_id=RWBXRNQ2JMFJ4SYHSVGRJPUYUQMQN2URNCN6LJFW3PDVS6EVDY4Q, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168095995 (314h44m9.599176126s) [noise=true , address=104.36.82.5:4843 , piece_id=Z7WZ5UTYX32KJADY3FSEJGPAKCNG7LU5FEPXZ4TAXXOGFQ4EYBEA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168341745 (314h44m9.599421866s) [noise=true , address=45.91.116.12:4255 , piece_id=TNSZHSIHY7JDMMW6D3TL42AFSE2OPVVAK2NMK6C52VBWMHUHPQ7A, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168407420 (314h44m9.599487551s) [noise=true , address=79.127.201.215:4344 , piece_id=2JUCTHQ3BI3B3SE3AQFC63L3GNLYMYHTWLRRHLDXC3CCTHS5EGKA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168667458 (314h44m9.599747579s) [noise=true , address=45.91.116.10:4114 , piece_id=REEPZ6DH3SKWEHY2N67ZCVVCOPSEZEFT366BC4JVBI7GYJ4WKIGQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168907448 (314h44m9.599987569s) [noise=true , address=77.90.31.12:4594 , piece_id=WDHXZIEPQRHWZHTWJ6CGMIRNIQAI3NZ7YVP2IICUGBWS3SUB77FA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168965298 (314h44m9.600045429s) [noise=true , address=77.90.31.11:4559 , piece_id=6NWPHRGM3VPBZG7IV5755VGSMSG5KD2S6FQYS7RJZ63N52L7UBDQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169160421 (314h44m9.600240553s) [noise=true , address=104.36.82.34:4570 , piece_id=SG6AVXZ6JMY6BZMPO7YO4D2M4GSKMHFYQJKBHQFNXB2OWXX7PKXA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169288357 (314h44m9.600368488s) [noise=true , address=104.36.82.13:4212 , piece_id=YYDY7XTFA6EGFPWTPSFRIDSRJYTJLTKM54UIDQUYN5ZM7OVPCK5Q, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169361095 (314h44m9.600441216s) [noise=true , address=104.36.82.23:4477 , piece_id=LQWY3THAYVTLQQWDKSM445MVXVDF2DQCEDLBROM7DONFMXSSSAVA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169555206 (314h44m9.600635337s) [noise=true , address=45.91.116.7:4014 , piece_id=SPFLBQZCPK33TXQ6IBHFCB3WH7XO4PHREI5MWVPNCEQF4EYZP5RQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169563844 (314h44m9.600643975s) [noise=true , address=104.36.82.40:4963 , piece_id=6VIZW4MFPEOVQORZYUS2VOAA6Q4U66ZWZEHFXKILJTSRF2RQLNSA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169877724 (314h44m9.600957845s) [noise=true , address=104.36.82.9:4018 , piece_id=7S4G6XMWKXQ5U3FQDMW2VDLZYWFOBVRKMCFED6BCCIWE4QAA7I2A, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.170176706 (314h44m9.601256837s) [noise=true , address=104.36.82.2:4762 , piece_id=OIUTGZSMAVR2HYTRWWMWLSR5XTXJQHFXWA6D2WU5JDNBXLRYWCOA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:16.672535880 (314h44m11.103616011s) [upload=327443]: Next returned nil segment Aug 21 20:10:16.672555797 (314h44m11.103635928s) [upload=327443]: Waiting for error group managing segments... Aug 21 20:10:17.083344442 (314h44m11.514424563s) [noise=true , address=45.91.116.7:4014 , piece_id=SPFLBQZCPK33TXQ6IBHFCB3WH7XO4PHREI5MWVPNCEQF4EYZP5RQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.083887000 (314h44m11.514967132s) [noise=true , address=45.91.116.4:4962 , piece_id=7W2H6URV32LWUBDIDSMFL4HS6DRZ5GFQAU6G25Y3PNJHYDIVCPZA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.090288509 (314h44m11.52136868s) [noise=true , address=77.90.31.17:4785 , piece_id=XKOCSZKVZ63UDGWWXP6DO6FVAWHNUSZLQZ75AYL72JSFA3LND7PQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.141165807 (314h44m11.572245938s) [noise=true , address=104.36.82.9:4018 , piece_id=7S4G6XMWKXQ5U3FQDMW2VDLZYWFOBVRKMCFED6BCCIWE4QAA7I2A, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.157740550 (314h44m11.588820682s) [noise=true , address=45.91.116.12:4255 , piece_id=TNSZHSIHY7JDMMW6D3TL42AFSE2OPVVAK2NMK6C52VBWMHUHPQ7A, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.158401245 (314h44m11.589481376s) [noise=true , address=77.90.31.11:4559 , piece_id=6NWPHRGM3VPBZG7IV5755VGSMSG5KD2S6FQYS7RJZ63N52L7UBDQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.170813964 (314h44m11.601894085s) [noise=true , address=45.91.116.10:4114 , piece_id=REEPZ6DH3SKWEHY2N67ZCVVCOPSEZEFT366BC4JVBI7GYJ4WKIGQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.175584983 (314h44m11.606665114s) [noise=true , address=79.127.201.215:4344 , piece_id=2JUCTHQ3BI3B3SE3AQFC63L3GNLYMYHTWLRRHLDXC3CCTHS5EGKA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.197949946 (314h44m11.629030077s) [noise=true , address=77.90.31.12:4594 , piece_id=WDHXZIEPQRHWZHTWJ6CGMIRNIQAI3NZ7YVP2IICUGBWS3SUB77FA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.202226388 (314h44m11.633306509s) [noise=true , address=104.36.82.2:4762 , piece_id=OIUTGZSMAVR2HYTRWWMWLSR5XTXJQHFXWA6D2WU5JDNBXLRYWCOA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.202498088 (314h44m11.633578209s) [noise=true , address=104.36.82.11:4126 , piece_id=246UC4PSMOZ2Q7557TLR4QUON7X2DW7SYQLH7534XSEDRKFK6YTA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.204637111 (314h44m11.635717232s) [noise=true , address=104.36.82.34:4570 , piece_id=SG6AVXZ6JMY6BZMPO7YO4D2M4GSKMHFYQJKBHQFNXB2OWXX7PKXA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.208072624 (314h44m11.639152755s) [noise=true , address=104.36.82.40:4963 , piece_id=6VIZW4MFPEOVQORZYUS2VOAA6Q4U66ZWZEHFXKILJTSRF2RQLNSA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.251397182 (314h44m11.682477313s) [noise=true , address=104.36.82.38:4713 , piece_id=RWBXRNQ2JMFJ4SYHSVGRJPUYUQMQN2URNCN6LJFW3PDVS6EVDY4Q, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.251419764 (314h44m11.682499895s) [seg_pos={0 0}, upload=327443]: Segment reached optimal threshold of 14 pieces. Aug 21 20:10:17.251567658 (314h44m11.682647789s) [noise=true , address=77.90.31.17:4775 , piece_id=ZGZNQUHXPKQMWYHUPES6JLK74YGWSOWSRVPTOCKEUQQGWNQEG2DQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127VKvwi82kYfGZ6Wcv8BinaNUQBMQsgKLKNNLht5JPsBLN7ovt): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251639294 (314h44m11.682719425s) [noise=true , address=104.36.82.23:4477 , piece_id=LQWY3THAYVTLQQWDKSM445MVXVDF2DQCEDLBROM7DONFMXSSSAVA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RP2Zq643WWNtKEr34Tq1iZS29NYA2okr9Fg1jiCpaFTqfLauf): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251720610 (314h44m11.682800741s) [noise=true , address=104.36.82.5:4843 , piece_id=Z7WZ5UTYX32KJADY3FSEJGPAKCNG7LU5FEPXZ4TAXXOGFQ4EYBEA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HvL2JDP4GCsu3tziVEKRMh14nEzyRMRuWNSmVvs27LzHTbNpS): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251764264 (314h44m11.682844395s) [noise=true , address=104.36.82.13:4212 , piece_id=YYDY7XTFA6EGFPWTPSFRIDSRJYTJLTKM54UIDQUYN5ZM7OVPCK5Q, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1itAUtWeFERY4AKhPxE2CRwymp8qNC6wu6uhPt2n6aK58LRu24): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251780665 (314h44m11.682860796s) [seg_pos={0 0}, upload=327443]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 21 20:10:17.251866870 (314h44m11.682947001s) [upload=327443]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 21 20:10:17.251871469 (314h44m11.6829516s) [upload=327443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:10:19.201398003 (314h44m13.632478134s) [upload=327443]: Done uploading segments... Aug 21 20:10:19.201416889 (314h44m13.63249702s) [upload=327443]: Upload finished. err: <nil> Duration: 5.794224901s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.781522839s FAIL (2025-08-21 20:15:21 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.781550483s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [................X...................................X.......] duration: 5.955981641s FAIL (2025-08-21 20:15:02 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 21 20:10:14.716582768 (314h44m9.147662899s) [upload=327443]: Starting upload Aug 21 20:10:14.716649245 (314h44m9.147729366s) [upload=327443]: Done starting upload Aug 21 20:10:14.716772782 (314h44m9.147852913s) [upload=327443]: Uploading segments... Aug 21 20:10:14.716914553 (314h44m9.147994684s) [upload=327443]: Got next segment. Inline: false Aug 21 20:10:14.716961734 (314h44m9.148041855s) [upload=327443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:10:14.716966182 (314h44m9.148046313s) [upload=327443]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 21 20:10:15.167114827 (314h44m9.598194958s) [seg_pos={0 0}, upload=327443]: Begin upload segment... Aug 21 20:10:15.167157349 (314h44m9.59823748s) [seg_pos={0 0}, upload=327443]: Done begin upload segment. Aug 21 20:10:15.167177778 (314h44m9.598257899s) [noise=true , address=77.90.31.17:4785 , piece_id=XKOCSZKVZ63UDGWWXP6DO6FVAWHNUSZLQZ75AYL72JSFA3LND7PQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167269874 (314h44m9.598349995s) [noise=true , address=45.91.116.4:4962 , piece_id=7W2H6URV32LWUBDIDSMFL4HS6DRZ5GFQAU6G25Y3PNJHYDIVCPZA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167549559 (314h44m9.59862969s) [noise=true , address=104.36.82.11:4126 , piece_id=246UC4PSMOZ2Q7557TLR4QUON7X2DW7SYQLH7534XSEDRKFK6YTA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167820397 (314h44m9.598900528s) [noise=true , address=77.90.31.17:4775 , piece_id=ZGZNQUHXPKQMWYHUPES6JLK74YGWSOWSRVPTOCKEUQQGWNQEG2DQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167897014 (314h44m9.598977135s) [noise=true , address=104.36.82.38:4713 , piece_id=RWBXRNQ2JMFJ4SYHSVGRJPUYUQMQN2URNCN6LJFW3PDVS6EVDY4Q, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168095995 (314h44m9.599176126s) [noise=true , address=104.36.82.5:4843 , piece_id=Z7WZ5UTYX32KJADY3FSEJGPAKCNG7LU5FEPXZ4TAXXOGFQ4EYBEA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168341745 (314h44m9.599421866s) [noise=true , address=45.91.116.12:4255 , piece_id=TNSZHSIHY7JDMMW6D3TL42AFSE2OPVVAK2NMK6C52VBWMHUHPQ7A, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168407420 (314h44m9.599487551s) [noise=true , address=79.127.201.215:4344 , piece_id=2JUCTHQ3BI3B3SE3AQFC63L3GNLYMYHTWLRRHLDXC3CCTHS5EGKA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168667458 (314h44m9.599747579s) [noise=true , address=45.91.116.10:4114 , piece_id=REEPZ6DH3SKWEHY2N67ZCVVCOPSEZEFT366BC4JVBI7GYJ4WKIGQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168907448 (314h44m9.599987569s) [noise=true , address=77.90.31.12:4594 , piece_id=WDHXZIEPQRHWZHTWJ6CGMIRNIQAI3NZ7YVP2IICUGBWS3SUB77FA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168965298 (314h44m9.600045429s) [noise=true , address=77.90.31.11:4559 , piece_id=6NWPHRGM3VPBZG7IV5755VGSMSG5KD2S6FQYS7RJZ63N52L7UBDQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169160421 (314h44m9.600240553s) [noise=true , address=104.36.82.34:4570 , piece_id=SG6AVXZ6JMY6BZMPO7YO4D2M4GSKMHFYQJKBHQFNXB2OWXX7PKXA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169288357 (314h44m9.600368488s) [noise=true , address=104.36.82.13:4212 , piece_id=YYDY7XTFA6EGFPWTPSFRIDSRJYTJLTKM54UIDQUYN5ZM7OVPCK5Q, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169361095 (314h44m9.600441216s) [noise=true , address=104.36.82.23:4477 , piece_id=LQWY3THAYVTLQQWDKSM445MVXVDF2DQCEDLBROM7DONFMXSSSAVA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169555206 (314h44m9.600635337s) [noise=true , address=45.91.116.7:4014 , piece_id=SPFLBQZCPK33TXQ6IBHFCB3WH7XO4PHREI5MWVPNCEQF4EYZP5RQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169563844 (314h44m9.600643975s) [noise=true , address=104.36.82.40:4963 , piece_id=6VIZW4MFPEOVQORZYUS2VOAA6Q4U66ZWZEHFXKILJTSRF2RQLNSA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169877724 (314h44m9.600957845s) [noise=true , address=104.36.82.9:4018 , piece_id=7S4G6XMWKXQ5U3FQDMW2VDLZYWFOBVRKMCFED6BCCIWE4QAA7I2A, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.170176706 (314h44m9.601256837s) [noise=true , address=104.36.82.2:4762 , piece_id=OIUTGZSMAVR2HYTRWWMWLSR5XTXJQHFXWA6D2WU5JDNBXLRYWCOA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:16.672535880 (314h44m11.103616011s) [upload=327443]: Next returned nil segment Aug 21 20:10:16.672555797 (314h44m11.103635928s) [upload=327443]: Waiting for error group managing segments... Aug 21 20:10:17.083344442 (314h44m11.514424563s) [noise=true , address=45.91.116.7:4014 , piece_id=SPFLBQZCPK33TXQ6IBHFCB3WH7XO4PHREI5MWVPNCEQF4EYZP5RQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.083887000 (314h44m11.514967132s) [noise=true , address=45.91.116.4:4962 , piece_id=7W2H6URV32LWUBDIDSMFL4HS6DRZ5GFQAU6G25Y3PNJHYDIVCPZA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.090288509 (314h44m11.52136868s) [noise=true , address=77.90.31.17:4785 , piece_id=XKOCSZKVZ63UDGWWXP6DO6FVAWHNUSZLQZ75AYL72JSFA3LND7PQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.141165807 (314h44m11.572245938s) [noise=true , address=104.36.82.9:4018 , piece_id=7S4G6XMWKXQ5U3FQDMW2VDLZYWFOBVRKMCFED6BCCIWE4QAA7I2A, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.157740550 (314h44m11.588820682s) [noise=true , address=45.91.116.12:4255 , piece_id=TNSZHSIHY7JDMMW6D3TL42AFSE2OPVVAK2NMK6C52VBWMHUHPQ7A, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.158401245 (314h44m11.589481376s) [noise=true , address=77.90.31.11:4559 , piece_id=6NWPHRGM3VPBZG7IV5755VGSMSG5KD2S6FQYS7RJZ63N52L7UBDQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.170813964 (314h44m11.601894085s) [noise=true , address=45.91.116.10:4114 , piece_id=REEPZ6DH3SKWEHY2N67ZCVVCOPSEZEFT366BC4JVBI7GYJ4WKIGQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.175584983 (314h44m11.606665114s) [noise=true , address=79.127.201.215:4344 , piece_id=2JUCTHQ3BI3B3SE3AQFC63L3GNLYMYHTWLRRHLDXC3CCTHS5EGKA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.197949946 (314h44m11.629030077s) [noise=true , address=77.90.31.12:4594 , piece_id=WDHXZIEPQRHWZHTWJ6CGMIRNIQAI3NZ7YVP2IICUGBWS3SUB77FA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.202226388 (314h44m11.633306509s) [noise=true , address=104.36.82.2:4762 , piece_id=OIUTGZSMAVR2HYTRWWMWLSR5XTXJQHFXWA6D2WU5JDNBXLRYWCOA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.202498088 (314h44m11.633578209s) [noise=true , address=104.36.82.11:4126 , piece_id=246UC4PSMOZ2Q7557TLR4QUON7X2DW7SYQLH7534XSEDRKFK6YTA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.204637111 (314h44m11.635717232s) [noise=true , address=104.36.82.34:4570 , piece_id=SG6AVXZ6JMY6BZMPO7YO4D2M4GSKMHFYQJKBHQFNXB2OWXX7PKXA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.208072624 (314h44m11.639152755s) [noise=true , address=104.36.82.40:4963 , piece_id=6VIZW4MFPEOVQORZYUS2VOAA6Q4U66ZWZEHFXKILJTSRF2RQLNSA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.251397182 (314h44m11.682477313s) [noise=true , address=104.36.82.38:4713 , piece_id=RWBXRNQ2JMFJ4SYHSVGRJPUYUQMQN2URNCN6LJFW3PDVS6EVDY4Q, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.251419764 (314h44m11.682499895s) [seg_pos={0 0}, upload=327443]: Segment reached optimal threshold of 14 pieces. Aug 21 20:10:17.251567658 (314h44m11.682647789s) [noise=true , address=77.90.31.17:4775 , piece_id=ZGZNQUHXPKQMWYHUPES6JLK74YGWSOWSRVPTOCKEUQQGWNQEG2DQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127VKvwi82kYfGZ6Wcv8BinaNUQBMQsgKLKNNLht5JPsBLN7ovt): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251639294 (314h44m11.682719425s) [noise=true , address=104.36.82.23:4477 , piece_id=LQWY3THAYVTLQQWDKSM445MVXVDF2DQCEDLBROM7DONFMXSSSAVA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RP2Zq643WWNtKEr34Tq1iZS29NYA2okr9Fg1jiCpaFTqfLauf): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251720610 (314h44m11.682800741s) [noise=true , address=104.36.82.5:4843 , piece_id=Z7WZ5UTYX32KJADY3FSEJGPAKCNG7LU5FEPXZ4TAXXOGFQ4EYBEA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HvL2JDP4GCsu3tziVEKRMh14nEzyRMRuWNSmVvs27LzHTbNpS): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251764264 (314h44m11.682844395s) [noise=true , address=104.36.82.13:4212 , piece_id=YYDY7XTFA6EGFPWTPSFRIDSRJYTJLTKM54UIDQUYN5ZM7OVPCK5Q, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1itAUtWeFERY4AKhPxE2CRwymp8qNC6wu6uhPt2n6aK58LRu24): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251780665 (314h44m11.682860796s) [seg_pos={0 0}, upload=327443]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 21 20:10:17.251866870 (314h44m11.682947001s) [upload=327443]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 21 20:10:17.251871469 (314h44m11.6829516s) [upload=327443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:10:19.201398003 (314h44m13.632478134s) [upload=327443]: Done uploading segments... Aug 21 20:10:19.201416889 (314h44m13.63249702s) [upload=327443]: Upload finished. err: <nil> Duration: 5.95600722s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............X..............................X................] duration: 2.001768613s FAIL (2025-08-21 20:14:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 20:05:21.070335328 (314h39m15.50141547s) [upload=327359]: Starting upload Aug 21 20:05:21.070433486 (314h39m15.501513657s) [upload=327359]: Done starting upload Aug 21 20:05:21.070504632 (314h39m15.501584773s) [upload=327359]: Uploading segments... Aug 21 20:05:21.070680819 (314h39m15.501761s) [upload=327359]: Got next segment. Inline: true Aug 21 20:05:21.070687241 (314h39m15.501767372s) [upload=327359]: Waiting for error group managing segments... Aug 21 20:05:21.070740553 (314h39m15.501820694s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:05:21.070747186 (314h39m15.501827338s) [upload=327359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 21 20:05:21.070754069 (314h39m15.501834221s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:05:21.283591166 (314h39m15.714671297s) [upload=327359]: Done uploading segments... Aug 21 20:05:21.283608600 (314h39m15.714688731s) [upload=327359]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.001792849s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................X...................................X......] duration: 5.840001284s FAIL (2025-08-21 20:14:31 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 21 20:10:14.716582768 (314h44m9.147662899s) [upload=327443]: Starting upload Aug 21 20:10:14.716649245 (314h44m9.147729366s) [upload=327443]: Done starting upload Aug 21 20:10:14.716772782 (314h44m9.147852913s) [upload=327443]: Uploading segments... Aug 21 20:10:14.716914553 (314h44m9.147994684s) [upload=327443]: Got next segment. Inline: false Aug 21 20:10:14.716961734 (314h44m9.148041855s) [upload=327443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:10:14.716966182 (314h44m9.148046313s) [upload=327443]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 21 20:10:15.167114827 (314h44m9.598194958s) [seg_pos={0 0}, upload=327443]: Begin upload segment... Aug 21 20:10:15.167157349 (314h44m9.59823748s) [seg_pos={0 0}, upload=327443]: Done begin upload segment. Aug 21 20:10:15.167177778 (314h44m9.598257899s) [noise=true , address=77.90.31.17:4785 , piece_id=XKOCSZKVZ63UDGWWXP6DO6FVAWHNUSZLQZ75AYL72JSFA3LND7PQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167269874 (314h44m9.598349995s) [noise=true , address=45.91.116.4:4962 , piece_id=7W2H6URV32LWUBDIDSMFL4HS6DRZ5GFQAU6G25Y3PNJHYDIVCPZA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167549559 (314h44m9.59862969s) [noise=true , address=104.36.82.11:4126 , piece_id=246UC4PSMOZ2Q7557TLR4QUON7X2DW7SYQLH7534XSEDRKFK6YTA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167820397 (314h44m9.598900528s) [noise=true , address=77.90.31.17:4775 , piece_id=ZGZNQUHXPKQMWYHUPES6JLK74YGWSOWSRVPTOCKEUQQGWNQEG2DQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167897014 (314h44m9.598977135s) [noise=true , address=104.36.82.38:4713 , piece_id=RWBXRNQ2JMFJ4SYHSVGRJPUYUQMQN2URNCN6LJFW3PDVS6EVDY4Q, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168095995 (314h44m9.599176126s) [noise=true , address=104.36.82.5:4843 , piece_id=Z7WZ5UTYX32KJADY3FSEJGPAKCNG7LU5FEPXZ4TAXXOGFQ4EYBEA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168341745 (314h44m9.599421866s) [noise=true , address=45.91.116.12:4255 , piece_id=TNSZHSIHY7JDMMW6D3TL42AFSE2OPVVAK2NMK6C52VBWMHUHPQ7A, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168407420 (314h44m9.599487551s) [noise=true , address=79.127.201.215:4344 , piece_id=2JUCTHQ3BI3B3SE3AQFC63L3GNLYMYHTWLRRHLDXC3CCTHS5EGKA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168667458 (314h44m9.599747579s) [noise=true , address=45.91.116.10:4114 , piece_id=REEPZ6DH3SKWEHY2N67ZCVVCOPSEZEFT366BC4JVBI7GYJ4WKIGQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168907448 (314h44m9.599987569s) [noise=true , address=77.90.31.12:4594 , piece_id=WDHXZIEPQRHWZHTWJ6CGMIRNIQAI3NZ7YVP2IICUGBWS3SUB77FA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168965298 (314h44m9.600045429s) [noise=true , address=77.90.31.11:4559 , piece_id=6NWPHRGM3VPBZG7IV5755VGSMSG5KD2S6FQYS7RJZ63N52L7UBDQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169160421 (314h44m9.600240553s) [noise=true , address=104.36.82.34:4570 , piece_id=SG6AVXZ6JMY6BZMPO7YO4D2M4GSKMHFYQJKBHQFNXB2OWXX7PKXA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169288357 (314h44m9.600368488s) [noise=true , address=104.36.82.13:4212 , piece_id=YYDY7XTFA6EGFPWTPSFRIDSRJYTJLTKM54UIDQUYN5ZM7OVPCK5Q, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169361095 (314h44m9.600441216s) [noise=true , address=104.36.82.23:4477 , piece_id=LQWY3THAYVTLQQWDKSM445MVXVDF2DQCEDLBROM7DONFMXSSSAVA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169555206 (314h44m9.600635337s) [noise=true , address=45.91.116.7:4014 , piece_id=SPFLBQZCPK33TXQ6IBHFCB3WH7XO4PHREI5MWVPNCEQF4EYZP5RQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169563844 (314h44m9.600643975s) [noise=true , address=104.36.82.40:4963 , piece_id=6VIZW4MFPEOVQORZYUS2VOAA6Q4U66ZWZEHFXKILJTSRF2RQLNSA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169877724 (314h44m9.600957845s) [noise=true , address=104.36.82.9:4018 , piece_id=7S4G6XMWKXQ5U3FQDMW2VDLZYWFOBVRKMCFED6BCCIWE4QAA7I2A, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.170176706 (314h44m9.601256837s) [noise=true , address=104.36.82.2:4762 , piece_id=OIUTGZSMAVR2HYTRWWMWLSR5XTXJQHFXWA6D2WU5JDNBXLRYWCOA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:16.672535880 (314h44m11.103616011s) [upload=327443]: Next returned nil segment Aug 21 20:10:16.672555797 (314h44m11.103635928s) [upload=327443]: Waiting for error group managing segments... Aug 21 20:10:17.083344442 (314h44m11.514424563s) [noise=true , address=45.91.116.7:4014 , piece_id=SPFLBQZCPK33TXQ6IBHFCB3WH7XO4PHREI5MWVPNCEQF4EYZP5RQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.083887000 (314h44m11.514967132s) [noise=true , address=45.91.116.4:4962 , piece_id=7W2H6URV32LWUBDIDSMFL4HS6DRZ5GFQAU6G25Y3PNJHYDIVCPZA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.090288509 (314h44m11.52136868s) [noise=true , address=77.90.31.17:4785 , piece_id=XKOCSZKVZ63UDGWWXP6DO6FVAWHNUSZLQZ75AYL72JSFA3LND7PQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.141165807 (314h44m11.572245938s) [noise=true , address=104.36.82.9:4018 , piece_id=7S4G6XMWKXQ5U3FQDMW2VDLZYWFOBVRKMCFED6BCCIWE4QAA7I2A, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.157740550 (314h44m11.588820682s) [noise=true , address=45.91.116.12:4255 , piece_id=TNSZHSIHY7JDMMW6D3TL42AFSE2OPVVAK2NMK6C52VBWMHUHPQ7A, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.158401245 (314h44m11.589481376s) [noise=true , address=77.90.31.11:4559 , piece_id=6NWPHRGM3VPBZG7IV5755VGSMSG5KD2S6FQYS7RJZ63N52L7UBDQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.170813964 (314h44m11.601894085s) [noise=true , address=45.91.116.10:4114 , piece_id=REEPZ6DH3SKWEHY2N67ZCVVCOPSEZEFT366BC4JVBI7GYJ4WKIGQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.175584983 (314h44m11.606665114s) [noise=true , address=79.127.201.215:4344 , piece_id=2JUCTHQ3BI3B3SE3AQFC63L3GNLYMYHTWLRRHLDXC3CCTHS5EGKA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.197949946 (314h44m11.629030077s) [noise=true , address=77.90.31.12:4594 , piece_id=WDHXZIEPQRHWZHTWJ6CGMIRNIQAI3NZ7YVP2IICUGBWS3SUB77FA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.202226388 (314h44m11.633306509s) [noise=true , address=104.36.82.2:4762 , piece_id=OIUTGZSMAVR2HYTRWWMWLSR5XTXJQHFXWA6D2WU5JDNBXLRYWCOA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.202498088 (314h44m11.633578209s) [noise=true , address=104.36.82.11:4126 , piece_id=246UC4PSMOZ2Q7557TLR4QUON7X2DW7SYQLH7534XSEDRKFK6YTA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.204637111 (314h44m11.635717232s) [noise=true , address=104.36.82.34:4570 , piece_id=SG6AVXZ6JMY6BZMPO7YO4D2M4GSKMHFYQJKBHQFNXB2OWXX7PKXA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.208072624 (314h44m11.639152755s) [noise=true , address=104.36.82.40:4963 , piece_id=6VIZW4MFPEOVQORZYUS2VOAA6Q4U66ZWZEHFXKILJTSRF2RQLNSA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.251397182 (314h44m11.682477313s) [noise=true , address=104.36.82.38:4713 , piece_id=RWBXRNQ2JMFJ4SYHSVGRJPUYUQMQN2URNCN6LJFW3PDVS6EVDY4Q, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.251419764 (314h44m11.682499895s) [seg_pos={0 0}, upload=327443]: Segment reached optimal threshold of 14 pieces. Aug 21 20:10:17.251567658 (314h44m11.682647789s) [noise=true , address=77.90.31.17:4775 , piece_id=ZGZNQUHXPKQMWYHUPES6JLK74YGWSOWSRVPTOCKEUQQGWNQEG2DQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127VKvwi82kYfGZ6Wcv8BinaNUQBMQsgKLKNNLht5JPsBLN7ovt): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251639294 (314h44m11.682719425s) [noise=true , address=104.36.82.23:4477 , piece_id=LQWY3THAYVTLQQWDKSM445MVXVDF2DQCEDLBROM7DONFMXSSSAVA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RP2Zq643WWNtKEr34Tq1iZS29NYA2okr9Fg1jiCpaFTqfLauf): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251720610 (314h44m11.682800741s) [noise=true , address=104.36.82.5:4843 , piece_id=Z7WZ5UTYX32KJADY3FSEJGPAKCNG7LU5FEPXZ4TAXXOGFQ4EYBEA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HvL2JDP4GCsu3tziVEKRMh14nEzyRMRuWNSmVvs27LzHTbNpS): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251764264 (314h44m11.682844395s) [noise=true , address=104.36.82.13:4212 , piece_id=YYDY7XTFA6EGFPWTPSFRIDSRJYTJLTKM54UIDQUYN5ZM7OVPCK5Q, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1itAUtWeFERY4AKhPxE2CRwymp8qNC6wu6uhPt2n6aK58LRu24): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251780665 (314h44m11.682860796s) [seg_pos={0 0}, upload=327443]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 21 20:10:17.251866870 (314h44m11.682947001s) [upload=327443]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 21 20:10:17.251871469 (314h44m11.6829516s) [upload=327443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:10:19.201398003 (314h44m13.632478134s) [upload=327443]: Done uploading segments... Aug 21 20:10:19.201416889 (314h44m13.63249702s) [upload=327443]: Upload finished. err: <nil> Duration: 5.840025191s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.107175802s FAIL (2025-08-21 20:14:26 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.107199527s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............X..............................X...............] duration: 2.019083931s FAIL (2025-08-21 20:14:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 20:05:21.070335328 (314h39m15.50141547s) [upload=327359]: Starting upload Aug 21 20:05:21.070433486 (314h39m15.501513657s) [upload=327359]: Done starting upload Aug 21 20:05:21.070504632 (314h39m15.501584773s) [upload=327359]: Uploading segments... Aug 21 20:05:21.070680819 (314h39m15.501761s) [upload=327359]: Got next segment. Inline: true Aug 21 20:05:21.070687241 (314h39m15.501767372s) [upload=327359]: Waiting for error group managing segments... Aug 21 20:05:21.070740553 (314h39m15.501820694s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:05:21.070747186 (314h39m15.501827338s) [upload=327359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 21 20:05:21.070754069 (314h39m15.501834221s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:05:21.283591166 (314h39m15.714671297s) [upload=327359]: Done uploading segments... Aug 21 20:05:21.283608600 (314h39m15.714688731s) [upload=327359]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.019110651s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............X..............................X..............] duration: 2.029380516s FAIL (2025-08-21 20:13:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 20:05:21.070335328 (314h39m15.50141547s) [upload=327359]: Starting upload Aug 21 20:05:21.070433486 (314h39m15.501513657s) [upload=327359]: Done starting upload Aug 21 20:05:21.070504632 (314h39m15.501584773s) [upload=327359]: Uploading segments... Aug 21 20:05:21.070680819 (314h39m15.501761s) [upload=327359]: Got next segment. Inline: true Aug 21 20:05:21.070687241 (314h39m15.501767372s) [upload=327359]: Waiting for error group managing segments... Aug 21 20:05:21.070740553 (314h39m15.501820694s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:05:21.070747186 (314h39m15.501827338s) [upload=327359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 21 20:05:21.070754069 (314h39m15.501834221s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:05:21.283591166 (314h39m15.714671297s) [upload=327359]: Done uploading segments... Aug 21 20:05:21.283608600 (314h39m15.714688731s) [upload=327359]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.029406547s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............X..............................X.............] duration: 1.92799201s FAIL (2025-08-21 20:13:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 20:05:21.070335328 (314h39m15.50141547s) [upload=327359]: Starting upload Aug 21 20:05:21.070433486 (314h39m15.501513657s) [upload=327359]: Done starting upload Aug 21 20:05:21.070504632 (314h39m15.501584773s) [upload=327359]: Uploading segments... Aug 21 20:05:21.070680819 (314h39m15.501761s) [upload=327359]: Got next segment. Inline: true Aug 21 20:05:21.070687241 (314h39m15.501767372s) [upload=327359]: Waiting for error group managing segments... Aug 21 20:05:21.070740553 (314h39m15.501820694s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:05:21.070747186 (314h39m15.501827338s) [upload=327359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 21 20:05:21.070754069 (314h39m15.501834221s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:05:21.283591166 (314h39m15.714671297s) [upload=327359]: Done uploading segments... Aug 21 20:05:21.283608600 (314h39m15.714688731s) [upload=327359]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.928014123s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.714046852s FAIL (2025-08-21 20:13:40 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.714062963s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [X.................X...................................X.....] duration: 5.962351993s FAIL (2025-08-21 20:13:25 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 21 20:10:14.716582768 (314h44m9.147662899s) [upload=327443]: Starting upload Aug 21 20:10:14.716649245 (314h44m9.147729366s) [upload=327443]: Done starting upload Aug 21 20:10:14.716772782 (314h44m9.147852913s) [upload=327443]: Uploading segments... Aug 21 20:10:14.716914553 (314h44m9.147994684s) [upload=327443]: Got next segment. Inline: false Aug 21 20:10:14.716961734 (314h44m9.148041855s) [upload=327443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:10:14.716966182 (314h44m9.148046313s) [upload=327443]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 21 20:10:15.167114827 (314h44m9.598194958s) [seg_pos={0 0}, upload=327443]: Begin upload segment... Aug 21 20:10:15.167157349 (314h44m9.59823748s) [seg_pos={0 0}, upload=327443]: Done begin upload segment. Aug 21 20:10:15.167177778 (314h44m9.598257899s) [noise=true , address=77.90.31.17:4785 , piece_id=XKOCSZKVZ63UDGWWXP6DO6FVAWHNUSZLQZ75AYL72JSFA3LND7PQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167269874 (314h44m9.598349995s) [noise=true , address=45.91.116.4:4962 , piece_id=7W2H6URV32LWUBDIDSMFL4HS6DRZ5GFQAU6G25Y3PNJHYDIVCPZA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167549559 (314h44m9.59862969s) [noise=true , address=104.36.82.11:4126 , piece_id=246UC4PSMOZ2Q7557TLR4QUON7X2DW7SYQLH7534XSEDRKFK6YTA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167820397 (314h44m9.598900528s) [noise=true , address=77.90.31.17:4775 , piece_id=ZGZNQUHXPKQMWYHUPES6JLK74YGWSOWSRVPTOCKEUQQGWNQEG2DQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167897014 (314h44m9.598977135s) [noise=true , address=104.36.82.38:4713 , piece_id=RWBXRNQ2JMFJ4SYHSVGRJPUYUQMQN2URNCN6LJFW3PDVS6EVDY4Q, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168095995 (314h44m9.599176126s) [noise=true , address=104.36.82.5:4843 , piece_id=Z7WZ5UTYX32KJADY3FSEJGPAKCNG7LU5FEPXZ4TAXXOGFQ4EYBEA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168341745 (314h44m9.599421866s) [noise=true , address=45.91.116.12:4255 , piece_id=TNSZHSIHY7JDMMW6D3TL42AFSE2OPVVAK2NMK6C52VBWMHUHPQ7A, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168407420 (314h44m9.599487551s) [noise=true , address=79.127.201.215:4344 , piece_id=2JUCTHQ3BI3B3SE3AQFC63L3GNLYMYHTWLRRHLDXC3CCTHS5EGKA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168667458 (314h44m9.599747579s) [noise=true , address=45.91.116.10:4114 , piece_id=REEPZ6DH3SKWEHY2N67ZCVVCOPSEZEFT366BC4JVBI7GYJ4WKIGQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168907448 (314h44m9.599987569s) [noise=true , address=77.90.31.12:4594 , piece_id=WDHXZIEPQRHWZHTWJ6CGMIRNIQAI3NZ7YVP2IICUGBWS3SUB77FA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168965298 (314h44m9.600045429s) [noise=true , address=77.90.31.11:4559 , piece_id=6NWPHRGM3VPBZG7IV5755VGSMSG5KD2S6FQYS7RJZ63N52L7UBDQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169160421 (314h44m9.600240553s) [noise=true , address=104.36.82.34:4570 , piece_id=SG6AVXZ6JMY6BZMPO7YO4D2M4GSKMHFYQJKBHQFNXB2OWXX7PKXA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169288357 (314h44m9.600368488s) [noise=true , address=104.36.82.13:4212 , piece_id=YYDY7XTFA6EGFPWTPSFRIDSRJYTJLTKM54UIDQUYN5ZM7OVPCK5Q, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169361095 (314h44m9.600441216s) [noise=true , address=104.36.82.23:4477 , piece_id=LQWY3THAYVTLQQWDKSM445MVXVDF2DQCEDLBROM7DONFMXSSSAVA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169555206 (314h44m9.600635337s) [noise=true , address=45.91.116.7:4014 , piece_id=SPFLBQZCPK33TXQ6IBHFCB3WH7XO4PHREI5MWVPNCEQF4EYZP5RQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169563844 (314h44m9.600643975s) [noise=true , address=104.36.82.40:4963 , piece_id=6VIZW4MFPEOVQORZYUS2VOAA6Q4U66ZWZEHFXKILJTSRF2RQLNSA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169877724 (314h44m9.600957845s) [noise=true , address=104.36.82.9:4018 , piece_id=7S4G6XMWKXQ5U3FQDMW2VDLZYWFOBVRKMCFED6BCCIWE4QAA7I2A, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.170176706 (314h44m9.601256837s) [noise=true , address=104.36.82.2:4762 , piece_id=OIUTGZSMAVR2HYTRWWMWLSR5XTXJQHFXWA6D2WU5JDNBXLRYWCOA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:16.672535880 (314h44m11.103616011s) [upload=327443]: Next returned nil segment Aug 21 20:10:16.672555797 (314h44m11.103635928s) [upload=327443]: Waiting for error group managing segments... Aug 21 20:10:17.083344442 (314h44m11.514424563s) [noise=true , address=45.91.116.7:4014 , piece_id=SPFLBQZCPK33TXQ6IBHFCB3WH7XO4PHREI5MWVPNCEQF4EYZP5RQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.083887000 (314h44m11.514967132s) [noise=true , address=45.91.116.4:4962 , piece_id=7W2H6URV32LWUBDIDSMFL4HS6DRZ5GFQAU6G25Y3PNJHYDIVCPZA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.090288509 (314h44m11.52136868s) [noise=true , address=77.90.31.17:4785 , piece_id=XKOCSZKVZ63UDGWWXP6DO6FVAWHNUSZLQZ75AYL72JSFA3LND7PQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.141165807 (314h44m11.572245938s) [noise=true , address=104.36.82.9:4018 , piece_id=7S4G6XMWKXQ5U3FQDMW2VDLZYWFOBVRKMCFED6BCCIWE4QAA7I2A, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.157740550 (314h44m11.588820682s) [noise=true , address=45.91.116.12:4255 , piece_id=TNSZHSIHY7JDMMW6D3TL42AFSE2OPVVAK2NMK6C52VBWMHUHPQ7A, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.158401245 (314h44m11.589481376s) [noise=true , address=77.90.31.11:4559 , piece_id=6NWPHRGM3VPBZG7IV5755VGSMSG5KD2S6FQYS7RJZ63N52L7UBDQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.170813964 (314h44m11.601894085s) [noise=true , address=45.91.116.10:4114 , piece_id=REEPZ6DH3SKWEHY2N67ZCVVCOPSEZEFT366BC4JVBI7GYJ4WKIGQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.175584983 (314h44m11.606665114s) [noise=true , address=79.127.201.215:4344 , piece_id=2JUCTHQ3BI3B3SE3AQFC63L3GNLYMYHTWLRRHLDXC3CCTHS5EGKA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.197949946 (314h44m11.629030077s) [noise=true , address=77.90.31.12:4594 , piece_id=WDHXZIEPQRHWZHTWJ6CGMIRNIQAI3NZ7YVP2IICUGBWS3SUB77FA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.202226388 (314h44m11.633306509s) [noise=true , address=104.36.82.2:4762 , piece_id=OIUTGZSMAVR2HYTRWWMWLSR5XTXJQHFXWA6D2WU5JDNBXLRYWCOA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.202498088 (314h44m11.633578209s) [noise=true , address=104.36.82.11:4126 , piece_id=246UC4PSMOZ2Q7557TLR4QUON7X2DW7SYQLH7534XSEDRKFK6YTA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.204637111 (314h44m11.635717232s) [noise=true , address=104.36.82.34:4570 , piece_id=SG6AVXZ6JMY6BZMPO7YO4D2M4GSKMHFYQJKBHQFNXB2OWXX7PKXA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.208072624 (314h44m11.639152755s) [noise=true , address=104.36.82.40:4963 , piece_id=6VIZW4MFPEOVQORZYUS2VOAA6Q4U66ZWZEHFXKILJTSRF2RQLNSA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.251397182 (314h44m11.682477313s) [noise=true , address=104.36.82.38:4713 , piece_id=RWBXRNQ2JMFJ4SYHSVGRJPUYUQMQN2URNCN6LJFW3PDVS6EVDY4Q, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.251419764 (314h44m11.682499895s) [seg_pos={0 0}, upload=327443]: Segment reached optimal threshold of 14 pieces. Aug 21 20:10:17.251567658 (314h44m11.682647789s) [noise=true , address=77.90.31.17:4775 , piece_id=ZGZNQUHXPKQMWYHUPES6JLK74YGWSOWSRVPTOCKEUQQGWNQEG2DQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127VKvwi82kYfGZ6Wcv8BinaNUQBMQsgKLKNNLht5JPsBLN7ovt): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251639294 (314h44m11.682719425s) [noise=true , address=104.36.82.23:4477 , piece_id=LQWY3THAYVTLQQWDKSM445MVXVDF2DQCEDLBROM7DONFMXSSSAVA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RP2Zq643WWNtKEr34Tq1iZS29NYA2okr9Fg1jiCpaFTqfLauf): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251720610 (314h44m11.682800741s) [noise=true , address=104.36.82.5:4843 , piece_id=Z7WZ5UTYX32KJADY3FSEJGPAKCNG7LU5FEPXZ4TAXXOGFQ4EYBEA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HvL2JDP4GCsu3tziVEKRMh14nEzyRMRuWNSmVvs27LzHTbNpS): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251764264 (314h44m11.682844395s) [noise=true , address=104.36.82.13:4212 , piece_id=YYDY7XTFA6EGFPWTPSFRIDSRJYTJLTKM54UIDQUYN5ZM7OVPCK5Q, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1itAUtWeFERY4AKhPxE2CRwymp8qNC6wu6uhPt2n6aK58LRu24): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251780665 (314h44m11.682860796s) [seg_pos={0 0}, upload=327443]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 21 20:10:17.251866870 (314h44m11.682947001s) [upload=327443]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 21 20:10:17.251871469 (314h44m11.6829516s) [upload=327443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:10:19.201398003 (314h44m13.632478134s) [upload=327443]: Done uploading segments... Aug 21 20:10:19.201416889 (314h44m13.63249702s) [upload=327443]: Upload finished. err: <nil> Duration: 5.962418129s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [................X..............................X............] duration: 2.23324897s FAIL (2025-08-21 20:12:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 20:05:21.070335328 (314h39m15.50141547s) [upload=327359]: Starting upload Aug 21 20:05:21.070433486 (314h39m15.501513657s) [upload=327359]: Done starting upload Aug 21 20:05:21.070504632 (314h39m15.501584773s) [upload=327359]: Uploading segments... Aug 21 20:05:21.070680819 (314h39m15.501761s) [upload=327359]: Got next segment. Inline: true Aug 21 20:05:21.070687241 (314h39m15.501767372s) [upload=327359]: Waiting for error group managing segments... Aug 21 20:05:21.070740553 (314h39m15.501820694s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:05:21.070747186 (314h39m15.501827338s) [upload=327359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 21 20:05:21.070754069 (314h39m15.501834221s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:05:21.283591166 (314h39m15.714671297s) [upload=327359]: Done uploading segments... Aug 21 20:05:21.283608600 (314h39m15.714688731s) [upload=327359]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.233272545s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.80989965s FAIL (2025-08-21 20:12:56 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.809927203s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X.................X...................................X....] duration: 5.904271729s FAIL (2025-08-21 20:12:56 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 21 20:10:14.716582768 (314h44m9.147662899s) [upload=327443]: Starting upload Aug 21 20:10:14.716649245 (314h44m9.147729366s) [upload=327443]: Done starting upload Aug 21 20:10:14.716772782 (314h44m9.147852913s) [upload=327443]: Uploading segments... Aug 21 20:10:14.716914553 (314h44m9.147994684s) [upload=327443]: Got next segment. Inline: false Aug 21 20:10:14.716961734 (314h44m9.148041855s) [upload=327443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:10:14.716966182 (314h44m9.148046313s) [upload=327443]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 21 20:10:15.167114827 (314h44m9.598194958s) [seg_pos={0 0}, upload=327443]: Begin upload segment... Aug 21 20:10:15.167157349 (314h44m9.59823748s) [seg_pos={0 0}, upload=327443]: Done begin upload segment. Aug 21 20:10:15.167177778 (314h44m9.598257899s) [noise=true , address=77.90.31.17:4785 , piece_id=XKOCSZKVZ63UDGWWXP6DO6FVAWHNUSZLQZ75AYL72JSFA3LND7PQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167269874 (314h44m9.598349995s) [noise=true , address=45.91.116.4:4962 , piece_id=7W2H6URV32LWUBDIDSMFL4HS6DRZ5GFQAU6G25Y3PNJHYDIVCPZA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167549559 (314h44m9.59862969s) [noise=true , address=104.36.82.11:4126 , piece_id=246UC4PSMOZ2Q7557TLR4QUON7X2DW7SYQLH7534XSEDRKFK6YTA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167820397 (314h44m9.598900528s) [noise=true , address=77.90.31.17:4775 , piece_id=ZGZNQUHXPKQMWYHUPES6JLK74YGWSOWSRVPTOCKEUQQGWNQEG2DQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167897014 (314h44m9.598977135s) [noise=true , address=104.36.82.38:4713 , piece_id=RWBXRNQ2JMFJ4SYHSVGRJPUYUQMQN2URNCN6LJFW3PDVS6EVDY4Q, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168095995 (314h44m9.599176126s) [noise=true , address=104.36.82.5:4843 , piece_id=Z7WZ5UTYX32KJADY3FSEJGPAKCNG7LU5FEPXZ4TAXXOGFQ4EYBEA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168341745 (314h44m9.599421866s) [noise=true , address=45.91.116.12:4255 , piece_id=TNSZHSIHY7JDMMW6D3TL42AFSE2OPVVAK2NMK6C52VBWMHUHPQ7A, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168407420 (314h44m9.599487551s) [noise=true , address=79.127.201.215:4344 , piece_id=2JUCTHQ3BI3B3SE3AQFC63L3GNLYMYHTWLRRHLDXC3CCTHS5EGKA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168667458 (314h44m9.599747579s) [noise=true , address=45.91.116.10:4114 , piece_id=REEPZ6DH3SKWEHY2N67ZCVVCOPSEZEFT366BC4JVBI7GYJ4WKIGQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168907448 (314h44m9.599987569s) [noise=true , address=77.90.31.12:4594 , piece_id=WDHXZIEPQRHWZHTWJ6CGMIRNIQAI3NZ7YVP2IICUGBWS3SUB77FA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168965298 (314h44m9.600045429s) [noise=true , address=77.90.31.11:4559 , piece_id=6NWPHRGM3VPBZG7IV5755VGSMSG5KD2S6FQYS7RJZ63N52L7UBDQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169160421 (314h44m9.600240553s) [noise=true , address=104.36.82.34:4570 , piece_id=SG6AVXZ6JMY6BZMPO7YO4D2M4GSKMHFYQJKBHQFNXB2OWXX7PKXA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169288357 (314h44m9.600368488s) [noise=true , address=104.36.82.13:4212 , piece_id=YYDY7XTFA6EGFPWTPSFRIDSRJYTJLTKM54UIDQUYN5ZM7OVPCK5Q, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169361095 (314h44m9.600441216s) [noise=true , address=104.36.82.23:4477 , piece_id=LQWY3THAYVTLQQWDKSM445MVXVDF2DQCEDLBROM7DONFMXSSSAVA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169555206 (314h44m9.600635337s) [noise=true , address=45.91.116.7:4014 , piece_id=SPFLBQZCPK33TXQ6IBHFCB3WH7XO4PHREI5MWVPNCEQF4EYZP5RQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169563844 (314h44m9.600643975s) [noise=true , address=104.36.82.40:4963 , piece_id=6VIZW4MFPEOVQORZYUS2VOAA6Q4U66ZWZEHFXKILJTSRF2RQLNSA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169877724 (314h44m9.600957845s) [noise=true , address=104.36.82.9:4018 , piece_id=7S4G6XMWKXQ5U3FQDMW2VDLZYWFOBVRKMCFED6BCCIWE4QAA7I2A, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.170176706 (314h44m9.601256837s) [noise=true , address=104.36.82.2:4762 , piece_id=OIUTGZSMAVR2HYTRWWMWLSR5XTXJQHFXWA6D2WU5JDNBXLRYWCOA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:16.672535880 (314h44m11.103616011s) [upload=327443]: Next returned nil segment Aug 21 20:10:16.672555797 (314h44m11.103635928s) [upload=327443]: Waiting for error group managing segments... Aug 21 20:10:17.083344442 (314h44m11.514424563s) [noise=true , address=45.91.116.7:4014 , piece_id=SPFLBQZCPK33TXQ6IBHFCB3WH7XO4PHREI5MWVPNCEQF4EYZP5RQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.083887000 (314h44m11.514967132s) [noise=true , address=45.91.116.4:4962 , piece_id=7W2H6URV32LWUBDIDSMFL4HS6DRZ5GFQAU6G25Y3PNJHYDIVCPZA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.090288509 (314h44m11.52136868s) [noise=true , address=77.90.31.17:4785 , piece_id=XKOCSZKVZ63UDGWWXP6DO6FVAWHNUSZLQZ75AYL72JSFA3LND7PQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.141165807 (314h44m11.572245938s) [noise=true , address=104.36.82.9:4018 , piece_id=7S4G6XMWKXQ5U3FQDMW2VDLZYWFOBVRKMCFED6BCCIWE4QAA7I2A, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.157740550 (314h44m11.588820682s) [noise=true , address=45.91.116.12:4255 , piece_id=TNSZHSIHY7JDMMW6D3TL42AFSE2OPVVAK2NMK6C52VBWMHUHPQ7A, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.158401245 (314h44m11.589481376s) [noise=true , address=77.90.31.11:4559 , piece_id=6NWPHRGM3VPBZG7IV5755VGSMSG5KD2S6FQYS7RJZ63N52L7UBDQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.170813964 (314h44m11.601894085s) [noise=true , address=45.91.116.10:4114 , piece_id=REEPZ6DH3SKWEHY2N67ZCVVCOPSEZEFT366BC4JVBI7GYJ4WKIGQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.175584983 (314h44m11.606665114s) [noise=true , address=79.127.201.215:4344 , piece_id=2JUCTHQ3BI3B3SE3AQFC63L3GNLYMYHTWLRRHLDXC3CCTHS5EGKA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.197949946 (314h44m11.629030077s) [noise=true , address=77.90.31.12:4594 , piece_id=WDHXZIEPQRHWZHTWJ6CGMIRNIQAI3NZ7YVP2IICUGBWS3SUB77FA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.202226388 (314h44m11.633306509s) [noise=true , address=104.36.82.2:4762 , piece_id=OIUTGZSMAVR2HYTRWWMWLSR5XTXJQHFXWA6D2WU5JDNBXLRYWCOA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.202498088 (314h44m11.633578209s) [noise=true , address=104.36.82.11:4126 , piece_id=246UC4PSMOZ2Q7557TLR4QUON7X2DW7SYQLH7534XSEDRKFK6YTA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.204637111 (314h44m11.635717232s) [noise=true , address=104.36.82.34:4570 , piece_id=SG6AVXZ6JMY6BZMPO7YO4D2M4GSKMHFYQJKBHQFNXB2OWXX7PKXA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.208072624 (314h44m11.639152755s) [noise=true , address=104.36.82.40:4963 , piece_id=6VIZW4MFPEOVQORZYUS2VOAA6Q4U66ZWZEHFXKILJTSRF2RQLNSA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.251397182 (314h44m11.682477313s) [noise=true , address=104.36.82.38:4713 , piece_id=RWBXRNQ2JMFJ4SYHSVGRJPUYUQMQN2URNCN6LJFW3PDVS6EVDY4Q, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.251419764 (314h44m11.682499895s) [seg_pos={0 0}, upload=327443]: Segment reached optimal threshold of 14 pieces. Aug 21 20:10:17.251567658 (314h44m11.682647789s) [noise=true , address=77.90.31.17:4775 , piece_id=ZGZNQUHXPKQMWYHUPES6JLK74YGWSOWSRVPTOCKEUQQGWNQEG2DQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127VKvwi82kYfGZ6Wcv8BinaNUQBMQsgKLKNNLht5JPsBLN7ovt): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251639294 (314h44m11.682719425s) [noise=true , address=104.36.82.23:4477 , piece_id=LQWY3THAYVTLQQWDKSM445MVXVDF2DQCEDLBROM7DONFMXSSSAVA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RP2Zq643WWNtKEr34Tq1iZS29NYA2okr9Fg1jiCpaFTqfLauf): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251720610 (314h44m11.682800741s) [noise=true , address=104.36.82.5:4843 , piece_id=Z7WZ5UTYX32KJADY3FSEJGPAKCNG7LU5FEPXZ4TAXXOGFQ4EYBEA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HvL2JDP4GCsu3tziVEKRMh14nEzyRMRuWNSmVvs27LzHTbNpS): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251764264 (314h44m11.682844395s) [noise=true , address=104.36.82.13:4212 , piece_id=YYDY7XTFA6EGFPWTPSFRIDSRJYTJLTKM54UIDQUYN5ZM7OVPCK5Q, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1itAUtWeFERY4AKhPxE2CRwymp8qNC6wu6uhPt2n6aK58LRu24): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251780665 (314h44m11.682860796s) [seg_pos={0 0}, upload=327443]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 21 20:10:17.251866870 (314h44m11.682947001s) [upload=327443]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 21 20:10:17.251871469 (314h44m11.6829516s) [upload=327443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:10:19.201398003 (314h44m13.632478134s) [upload=327443]: Done uploading segments... Aug 21 20:10:19.201416889 (314h44m13.63249702s) [upload=327443]: Upload finished. err: <nil> Duration: 5.904292118s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.853981983s FAIL (2025-08-21 20:12:26 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.854006821s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.................X..............................X...........] duration: 2.290271533s FAIL (2025-08-21 20:12:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 20:05:21.070335328 (314h39m15.50141547s) [upload=327359]: Starting upload Aug 21 20:05:21.070433486 (314h39m15.501513657s) [upload=327359]: Done starting upload Aug 21 20:05:21.070504632 (314h39m15.501584773s) [upload=327359]: Uploading segments... Aug 21 20:05:21.070680819 (314h39m15.501761s) [upload=327359]: Got next segment. Inline: true Aug 21 20:05:21.070687241 (314h39m15.501767372s) [upload=327359]: Waiting for error group managing segments... Aug 21 20:05:21.070740553 (314h39m15.501820694s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:05:21.070747186 (314h39m15.501827338s) [upload=327359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 21 20:05:21.070754069 (314h39m15.501834221s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:05:21.283591166 (314h39m15.714671297s) [upload=327359]: Done uploading segments... Aug 21 20:05:21.283608600 (314h39m15.714688731s) [upload=327359]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.290293406s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..X.................X...................................X...] duration: 5.884746763s FAIL (2025-08-21 20:12:02 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 21 20:10:14.716582768 (314h44m9.147662899s) [upload=327443]: Starting upload Aug 21 20:10:14.716649245 (314h44m9.147729366s) [upload=327443]: Done starting upload Aug 21 20:10:14.716772782 (314h44m9.147852913s) [upload=327443]: Uploading segments... Aug 21 20:10:14.716914553 (314h44m9.147994684s) [upload=327443]: Got next segment. Inline: false Aug 21 20:10:14.716961734 (314h44m9.148041855s) [upload=327443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:10:14.716966182 (314h44m9.148046313s) [upload=327443]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 21 20:10:15.167114827 (314h44m9.598194958s) [seg_pos={0 0}, upload=327443]: Begin upload segment... Aug 21 20:10:15.167157349 (314h44m9.59823748s) [seg_pos={0 0}, upload=327443]: Done begin upload segment. Aug 21 20:10:15.167177778 (314h44m9.598257899s) [noise=true , address=77.90.31.17:4785 , piece_id=XKOCSZKVZ63UDGWWXP6DO6FVAWHNUSZLQZ75AYL72JSFA3LND7PQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167269874 (314h44m9.598349995s) [noise=true , address=45.91.116.4:4962 , piece_id=7W2H6URV32LWUBDIDSMFL4HS6DRZ5GFQAU6G25Y3PNJHYDIVCPZA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167549559 (314h44m9.59862969s) [noise=true , address=104.36.82.11:4126 , piece_id=246UC4PSMOZ2Q7557TLR4QUON7X2DW7SYQLH7534XSEDRKFK6YTA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167820397 (314h44m9.598900528s) [noise=true , address=77.90.31.17:4775 , piece_id=ZGZNQUHXPKQMWYHUPES6JLK74YGWSOWSRVPTOCKEUQQGWNQEG2DQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167897014 (314h44m9.598977135s) [noise=true , address=104.36.82.38:4713 , piece_id=RWBXRNQ2JMFJ4SYHSVGRJPUYUQMQN2URNCN6LJFW3PDVS6EVDY4Q, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168095995 (314h44m9.599176126s) [noise=true , address=104.36.82.5:4843 , piece_id=Z7WZ5UTYX32KJADY3FSEJGPAKCNG7LU5FEPXZ4TAXXOGFQ4EYBEA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168341745 (314h44m9.599421866s) [noise=true , address=45.91.116.12:4255 , piece_id=TNSZHSIHY7JDMMW6D3TL42AFSE2OPVVAK2NMK6C52VBWMHUHPQ7A, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168407420 (314h44m9.599487551s) [noise=true , address=79.127.201.215:4344 , piece_id=2JUCTHQ3BI3B3SE3AQFC63L3GNLYMYHTWLRRHLDXC3CCTHS5EGKA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168667458 (314h44m9.599747579s) [noise=true , address=45.91.116.10:4114 , piece_id=REEPZ6DH3SKWEHY2N67ZCVVCOPSEZEFT366BC4JVBI7GYJ4WKIGQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168907448 (314h44m9.599987569s) [noise=true , address=77.90.31.12:4594 , piece_id=WDHXZIEPQRHWZHTWJ6CGMIRNIQAI3NZ7YVP2IICUGBWS3SUB77FA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168965298 (314h44m9.600045429s) [noise=true , address=77.90.31.11:4559 , piece_id=6NWPHRGM3VPBZG7IV5755VGSMSG5KD2S6FQYS7RJZ63N52L7UBDQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169160421 (314h44m9.600240553s) [noise=true , address=104.36.82.34:4570 , piece_id=SG6AVXZ6JMY6BZMPO7YO4D2M4GSKMHFYQJKBHQFNXB2OWXX7PKXA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169288357 (314h44m9.600368488s) [noise=true , address=104.36.82.13:4212 , piece_id=YYDY7XTFA6EGFPWTPSFRIDSRJYTJLTKM54UIDQUYN5ZM7OVPCK5Q, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169361095 (314h44m9.600441216s) [noise=true , address=104.36.82.23:4477 , piece_id=LQWY3THAYVTLQQWDKSM445MVXVDF2DQCEDLBROM7DONFMXSSSAVA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169555206 (314h44m9.600635337s) [noise=true , address=45.91.116.7:4014 , piece_id=SPFLBQZCPK33TXQ6IBHFCB3WH7XO4PHREI5MWVPNCEQF4EYZP5RQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169563844 (314h44m9.600643975s) [noise=true , address=104.36.82.40:4963 , piece_id=6VIZW4MFPEOVQORZYUS2VOAA6Q4U66ZWZEHFXKILJTSRF2RQLNSA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169877724 (314h44m9.600957845s) [noise=true , address=104.36.82.9:4018 , piece_id=7S4G6XMWKXQ5U3FQDMW2VDLZYWFOBVRKMCFED6BCCIWE4QAA7I2A, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.170176706 (314h44m9.601256837s) [noise=true , address=104.36.82.2:4762 , piece_id=OIUTGZSMAVR2HYTRWWMWLSR5XTXJQHFXWA6D2WU5JDNBXLRYWCOA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:16.672535880 (314h44m11.103616011s) [upload=327443]: Next returned nil segment Aug 21 20:10:16.672555797 (314h44m11.103635928s) [upload=327443]: Waiting for error group managing segments... Aug 21 20:10:17.083344442 (314h44m11.514424563s) [noise=true , address=45.91.116.7:4014 , piece_id=SPFLBQZCPK33TXQ6IBHFCB3WH7XO4PHREI5MWVPNCEQF4EYZP5RQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.083887000 (314h44m11.514967132s) [noise=true , address=45.91.116.4:4962 , piece_id=7W2H6URV32LWUBDIDSMFL4HS6DRZ5GFQAU6G25Y3PNJHYDIVCPZA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.090288509 (314h44m11.52136868s) [noise=true , address=77.90.31.17:4785 , piece_id=XKOCSZKVZ63UDGWWXP6DO6FVAWHNUSZLQZ75AYL72JSFA3LND7PQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.141165807 (314h44m11.572245938s) [noise=true , address=104.36.82.9:4018 , piece_id=7S4G6XMWKXQ5U3FQDMW2VDLZYWFOBVRKMCFED6BCCIWE4QAA7I2A, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.157740550 (314h44m11.588820682s) [noise=true , address=45.91.116.12:4255 , piece_id=TNSZHSIHY7JDMMW6D3TL42AFSE2OPVVAK2NMK6C52VBWMHUHPQ7A, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.158401245 (314h44m11.589481376s) [noise=true , address=77.90.31.11:4559 , piece_id=6NWPHRGM3VPBZG7IV5755VGSMSG5KD2S6FQYS7RJZ63N52L7UBDQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.170813964 (314h44m11.601894085s) [noise=true , address=45.91.116.10:4114 , piece_id=REEPZ6DH3SKWEHY2N67ZCVVCOPSEZEFT366BC4JVBI7GYJ4WKIGQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.175584983 (314h44m11.606665114s) [noise=true , address=79.127.201.215:4344 , piece_id=2JUCTHQ3BI3B3SE3AQFC63L3GNLYMYHTWLRRHLDXC3CCTHS5EGKA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.197949946 (314h44m11.629030077s) [noise=true , address=77.90.31.12:4594 , piece_id=WDHXZIEPQRHWZHTWJ6CGMIRNIQAI3NZ7YVP2IICUGBWS3SUB77FA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.202226388 (314h44m11.633306509s) [noise=true , address=104.36.82.2:4762 , piece_id=OIUTGZSMAVR2HYTRWWMWLSR5XTXJQHFXWA6D2WU5JDNBXLRYWCOA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.202498088 (314h44m11.633578209s) [noise=true , address=104.36.82.11:4126 , piece_id=246UC4PSMOZ2Q7557TLR4QUON7X2DW7SYQLH7534XSEDRKFK6YTA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.204637111 (314h44m11.635717232s) [noise=true , address=104.36.82.34:4570 , piece_id=SG6AVXZ6JMY6BZMPO7YO4D2M4GSKMHFYQJKBHQFNXB2OWXX7PKXA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.208072624 (314h44m11.639152755s) [noise=true , address=104.36.82.40:4963 , piece_id=6VIZW4MFPEOVQORZYUS2VOAA6Q4U66ZWZEHFXKILJTSRF2RQLNSA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.251397182 (314h44m11.682477313s) [noise=true , address=104.36.82.38:4713 , piece_id=RWBXRNQ2JMFJ4SYHSVGRJPUYUQMQN2URNCN6LJFW3PDVS6EVDY4Q, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.251419764 (314h44m11.682499895s) [seg_pos={0 0}, upload=327443]: Segment reached optimal threshold of 14 pieces. Aug 21 20:10:17.251567658 (314h44m11.682647789s) [noise=true , address=77.90.31.17:4775 , piece_id=ZGZNQUHXPKQMWYHUPES6JLK74YGWSOWSRVPTOCKEUQQGWNQEG2DQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127VKvwi82kYfGZ6Wcv8BinaNUQBMQsgKLKNNLht5JPsBLN7ovt): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251639294 (314h44m11.682719425s) [noise=true , address=104.36.82.23:4477 , piece_id=LQWY3THAYVTLQQWDKSM445MVXVDF2DQCEDLBROM7DONFMXSSSAVA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RP2Zq643WWNtKEr34Tq1iZS29NYA2okr9Fg1jiCpaFTqfLauf): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251720610 (314h44m11.682800741s) [noise=true , address=104.36.82.5:4843 , piece_id=Z7WZ5UTYX32KJADY3FSEJGPAKCNG7LU5FEPXZ4TAXXOGFQ4EYBEA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HvL2JDP4GCsu3tziVEKRMh14nEzyRMRuWNSmVvs27LzHTbNpS): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251764264 (314h44m11.682844395s) [noise=true , address=104.36.82.13:4212 , piece_id=YYDY7XTFA6EGFPWTPSFRIDSRJYTJLTKM54UIDQUYN5ZM7OVPCK5Q, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1itAUtWeFERY4AKhPxE2CRwymp8qNC6wu6uhPt2n6aK58LRu24): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251780665 (314h44m11.682860796s) [seg_pos={0 0}, upload=327443]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 21 20:10:17.251866870 (314h44m11.682947001s) [upload=327443]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 21 20:10:17.251871469 (314h44m11.6829516s) [upload=327443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:10:19.201398003 (314h44m13.632478134s) [upload=327443]: Done uploading segments... Aug 21 20:10:19.201416889 (314h44m13.63249702s) [upload=327443]: Upload finished. err: <nil> Duration: 5.884774687s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.62217729s FAIL (2025-08-21 20:11:56 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.622211384s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..................X..............................X..........] duration: 1.897828826s FAIL (2025-08-21 20:11:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 20:05:21.070335328 (314h39m15.50141547s) [upload=327359]: Starting upload Aug 21 20:05:21.070433486 (314h39m15.501513657s) [upload=327359]: Done starting upload Aug 21 20:05:21.070504632 (314h39m15.501584773s) [upload=327359]: Uploading segments... Aug 21 20:05:21.070680819 (314h39m15.501761s) [upload=327359]: Got next segment. Inline: true Aug 21 20:05:21.070687241 (314h39m15.501767372s) [upload=327359]: Waiting for error group managing segments... Aug 21 20:05:21.070740553 (314h39m15.501820694s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:05:21.070747186 (314h39m15.501827338s) [upload=327359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 21 20:05:21.070754069 (314h39m15.501834221s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:05:21.283591166 (314h39m15.714671297s) [upload=327359]: Done uploading segments... Aug 21 20:05:21.283608600 (314h39m15.714688731s) [upload=327359]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.897858212s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X.................X...................................X..] duration: 5.628490888s FAIL (2025-08-21 20:11:31 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 21 20:10:14.716582768 (314h44m9.147662899s) [upload=327443]: Starting upload Aug 21 20:10:14.716649245 (314h44m9.147729366s) [upload=327443]: Done starting upload Aug 21 20:10:14.716772782 (314h44m9.147852913s) [upload=327443]: Uploading segments... Aug 21 20:10:14.716914553 (314h44m9.147994684s) [upload=327443]: Got next segment. Inline: false Aug 21 20:10:14.716961734 (314h44m9.148041855s) [upload=327443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:10:14.716966182 (314h44m9.148046313s) [upload=327443]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 21 20:10:15.167114827 (314h44m9.598194958s) [seg_pos={0 0}, upload=327443]: Begin upload segment... Aug 21 20:10:15.167157349 (314h44m9.59823748s) [seg_pos={0 0}, upload=327443]: Done begin upload segment. Aug 21 20:10:15.167177778 (314h44m9.598257899s) [noise=true , address=77.90.31.17:4785 , piece_id=XKOCSZKVZ63UDGWWXP6DO6FVAWHNUSZLQZ75AYL72JSFA3LND7PQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167269874 (314h44m9.598349995s) [noise=true , address=45.91.116.4:4962 , piece_id=7W2H6URV32LWUBDIDSMFL4HS6DRZ5GFQAU6G25Y3PNJHYDIVCPZA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167549559 (314h44m9.59862969s) [noise=true , address=104.36.82.11:4126 , piece_id=246UC4PSMOZ2Q7557TLR4QUON7X2DW7SYQLH7534XSEDRKFK6YTA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167820397 (314h44m9.598900528s) [noise=true , address=77.90.31.17:4775 , piece_id=ZGZNQUHXPKQMWYHUPES6JLK74YGWSOWSRVPTOCKEUQQGWNQEG2DQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167897014 (314h44m9.598977135s) [noise=true , address=104.36.82.38:4713 , piece_id=RWBXRNQ2JMFJ4SYHSVGRJPUYUQMQN2URNCN6LJFW3PDVS6EVDY4Q, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168095995 (314h44m9.599176126s) [noise=true , address=104.36.82.5:4843 , piece_id=Z7WZ5UTYX32KJADY3FSEJGPAKCNG7LU5FEPXZ4TAXXOGFQ4EYBEA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168341745 (314h44m9.599421866s) [noise=true , address=45.91.116.12:4255 , piece_id=TNSZHSIHY7JDMMW6D3TL42AFSE2OPVVAK2NMK6C52VBWMHUHPQ7A, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168407420 (314h44m9.599487551s) [noise=true , address=79.127.201.215:4344 , piece_id=2JUCTHQ3BI3B3SE3AQFC63L3GNLYMYHTWLRRHLDXC3CCTHS5EGKA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168667458 (314h44m9.599747579s) [noise=true , address=45.91.116.10:4114 , piece_id=REEPZ6DH3SKWEHY2N67ZCVVCOPSEZEFT366BC4JVBI7GYJ4WKIGQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168907448 (314h44m9.599987569s) [noise=true , address=77.90.31.12:4594 , piece_id=WDHXZIEPQRHWZHTWJ6CGMIRNIQAI3NZ7YVP2IICUGBWS3SUB77FA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168965298 (314h44m9.600045429s) [noise=true , address=77.90.31.11:4559 , piece_id=6NWPHRGM3VPBZG7IV5755VGSMSG5KD2S6FQYS7RJZ63N52L7UBDQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169160421 (314h44m9.600240553s) [noise=true , address=104.36.82.34:4570 , piece_id=SG6AVXZ6JMY6BZMPO7YO4D2M4GSKMHFYQJKBHQFNXB2OWXX7PKXA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169288357 (314h44m9.600368488s) [noise=true , address=104.36.82.13:4212 , piece_id=YYDY7XTFA6EGFPWTPSFRIDSRJYTJLTKM54UIDQUYN5ZM7OVPCK5Q, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169361095 (314h44m9.600441216s) [noise=true , address=104.36.82.23:4477 , piece_id=LQWY3THAYVTLQQWDKSM445MVXVDF2DQCEDLBROM7DONFMXSSSAVA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169555206 (314h44m9.600635337s) [noise=true , address=45.91.116.7:4014 , piece_id=SPFLBQZCPK33TXQ6IBHFCB3WH7XO4PHREI5MWVPNCEQF4EYZP5RQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169563844 (314h44m9.600643975s) [noise=true , address=104.36.82.40:4963 , piece_id=6VIZW4MFPEOVQORZYUS2VOAA6Q4U66ZWZEHFXKILJTSRF2RQLNSA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169877724 (314h44m9.600957845s) [noise=true , address=104.36.82.9:4018 , piece_id=7S4G6XMWKXQ5U3FQDMW2VDLZYWFOBVRKMCFED6BCCIWE4QAA7I2A, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.170176706 (314h44m9.601256837s) [noise=true , address=104.36.82.2:4762 , piece_id=OIUTGZSMAVR2HYTRWWMWLSR5XTXJQHFXWA6D2WU5JDNBXLRYWCOA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:16.672535880 (314h44m11.103616011s) [upload=327443]: Next returned nil segment Aug 21 20:10:16.672555797 (314h44m11.103635928s) [upload=327443]: Waiting for error group managing segments... Aug 21 20:10:17.083344442 (314h44m11.514424563s) [noise=true , address=45.91.116.7:4014 , piece_id=SPFLBQZCPK33TXQ6IBHFCB3WH7XO4PHREI5MWVPNCEQF4EYZP5RQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.083887000 (314h44m11.514967132s) [noise=true , address=45.91.116.4:4962 , piece_id=7W2H6URV32LWUBDIDSMFL4HS6DRZ5GFQAU6G25Y3PNJHYDIVCPZA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.090288509 (314h44m11.52136868s) [noise=true , address=77.90.31.17:4785 , piece_id=XKOCSZKVZ63UDGWWXP6DO6FVAWHNUSZLQZ75AYL72JSFA3LND7PQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.141165807 (314h44m11.572245938s) [noise=true , address=104.36.82.9:4018 , piece_id=7S4G6XMWKXQ5U3FQDMW2VDLZYWFOBVRKMCFED6BCCIWE4QAA7I2A, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.157740550 (314h44m11.588820682s) [noise=true , address=45.91.116.12:4255 , piece_id=TNSZHSIHY7JDMMW6D3TL42AFSE2OPVVAK2NMK6C52VBWMHUHPQ7A, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.158401245 (314h44m11.589481376s) [noise=true , address=77.90.31.11:4559 , piece_id=6NWPHRGM3VPBZG7IV5755VGSMSG5KD2S6FQYS7RJZ63N52L7UBDQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.170813964 (314h44m11.601894085s) [noise=true , address=45.91.116.10:4114 , piece_id=REEPZ6DH3SKWEHY2N67ZCVVCOPSEZEFT366BC4JVBI7GYJ4WKIGQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.175584983 (314h44m11.606665114s) [noise=true , address=79.127.201.215:4344 , piece_id=2JUCTHQ3BI3B3SE3AQFC63L3GNLYMYHTWLRRHLDXC3CCTHS5EGKA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.197949946 (314h44m11.629030077s) [noise=true , address=77.90.31.12:4594 , piece_id=WDHXZIEPQRHWZHTWJ6CGMIRNIQAI3NZ7YVP2IICUGBWS3SUB77FA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.202226388 (314h44m11.633306509s) [noise=true , address=104.36.82.2:4762 , piece_id=OIUTGZSMAVR2HYTRWWMWLSR5XTXJQHFXWA6D2WU5JDNBXLRYWCOA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.202498088 (314h44m11.633578209s) [noise=true , address=104.36.82.11:4126 , piece_id=246UC4PSMOZ2Q7557TLR4QUON7X2DW7SYQLH7534XSEDRKFK6YTA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.204637111 (314h44m11.635717232s) [noise=true , address=104.36.82.34:4570 , piece_id=SG6AVXZ6JMY6BZMPO7YO4D2M4GSKMHFYQJKBHQFNXB2OWXX7PKXA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.208072624 (314h44m11.639152755s) [noise=true , address=104.36.82.40:4963 , piece_id=6VIZW4MFPEOVQORZYUS2VOAA6Q4U66ZWZEHFXKILJTSRF2RQLNSA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.251397182 (314h44m11.682477313s) [noise=true , address=104.36.82.38:4713 , piece_id=RWBXRNQ2JMFJ4SYHSVGRJPUYUQMQN2URNCN6LJFW3PDVS6EVDY4Q, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.251419764 (314h44m11.682499895s) [seg_pos={0 0}, upload=327443]: Segment reached optimal threshold of 14 pieces. Aug 21 20:10:17.251567658 (314h44m11.682647789s) [noise=true , address=77.90.31.17:4775 , piece_id=ZGZNQUHXPKQMWYHUPES6JLK74YGWSOWSRVPTOCKEUQQGWNQEG2DQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127VKvwi82kYfGZ6Wcv8BinaNUQBMQsgKLKNNLht5JPsBLN7ovt): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251639294 (314h44m11.682719425s) [noise=true , address=104.36.82.23:4477 , piece_id=LQWY3THAYVTLQQWDKSM445MVXVDF2DQCEDLBROM7DONFMXSSSAVA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RP2Zq643WWNtKEr34Tq1iZS29NYA2okr9Fg1jiCpaFTqfLauf): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251720610 (314h44m11.682800741s) [noise=true , address=104.36.82.5:4843 , piece_id=Z7WZ5UTYX32KJADY3FSEJGPAKCNG7LU5FEPXZ4TAXXOGFQ4EYBEA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HvL2JDP4GCsu3tziVEKRMh14nEzyRMRuWNSmVvs27LzHTbNpS): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251764264 (314h44m11.682844395s) [noise=true , address=104.36.82.13:4212 , piece_id=YYDY7XTFA6EGFPWTPSFRIDSRJYTJLTKM54UIDQUYN5ZM7OVPCK5Q, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1itAUtWeFERY4AKhPxE2CRwymp8qNC6wu6uhPt2n6aK58LRu24): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251780665 (314h44m11.682860796s) [seg_pos={0 0}, upload=327443]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 21 20:10:17.251866870 (314h44m11.682947001s) [upload=327443]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 21 20:10:17.251871469 (314h44m11.6829516s) [upload=327443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:10:19.201398003 (314h44m13.632478134s) [upload=327443]: Done uploading segments... Aug 21 20:10:19.201416889 (314h44m13.63249702s) [upload=327443]: Upload finished. err: <nil> Duration: 5.628515456s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [X..................X..............................X.........] duration: 1.92242679s FAIL (2025-08-21 20:11:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 20:05:21.070335328 (314h39m15.50141547s) [upload=327359]: Starting upload Aug 21 20:05:21.070433486 (314h39m15.501513657s) [upload=327359]: Done starting upload Aug 21 20:05:21.070504632 (314h39m15.501584773s) [upload=327359]: Uploading segments... Aug 21 20:05:21.070680819 (314h39m15.501761s) [upload=327359]: Got next segment. Inline: true Aug 21 20:05:21.070687241 (314h39m15.501767372s) [upload=327359]: Waiting for error group managing segments... Aug 21 20:05:21.070740553 (314h39m15.501820694s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:05:21.070747186 (314h39m15.501827338s) [upload=327359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 21 20:05:21.070754069 (314h39m15.501834221s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:05:21.283591166 (314h39m15.714671297s) [upload=327359]: Done uploading segments... Aug 21 20:05:21.283608600 (314h39m15.714688731s) [upload=327359]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.922455307s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.040373896s FAIL (2025-08-21 20:11:10 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.040404264s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X.................X...................................X.] duration: 5.82326915s FAIL (2025-08-21 20:10:47 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 21 20:10:14.716582768 (314h44m9.147662899s) [upload=327443]: Starting upload Aug 21 20:10:14.716649245 (314h44m9.147729366s) [upload=327443]: Done starting upload Aug 21 20:10:14.716772782 (314h44m9.147852913s) [upload=327443]: Uploading segments... Aug 21 20:10:14.716914553 (314h44m9.147994684s) [upload=327443]: Got next segment. Inline: false Aug 21 20:10:14.716961734 (314h44m9.148041855s) [upload=327443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:10:14.716966182 (314h44m9.148046313s) [upload=327443]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 21 20:10:15.167114827 (314h44m9.598194958s) [seg_pos={0 0}, upload=327443]: Begin upload segment... Aug 21 20:10:15.167157349 (314h44m9.59823748s) [seg_pos={0 0}, upload=327443]: Done begin upload segment. Aug 21 20:10:15.167177778 (314h44m9.598257899s) [noise=true , address=77.90.31.17:4785 , piece_id=XKOCSZKVZ63UDGWWXP6DO6FVAWHNUSZLQZ75AYL72JSFA3LND7PQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167269874 (314h44m9.598349995s) [noise=true , address=45.91.116.4:4962 , piece_id=7W2H6URV32LWUBDIDSMFL4HS6DRZ5GFQAU6G25Y3PNJHYDIVCPZA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167549559 (314h44m9.59862969s) [noise=true , address=104.36.82.11:4126 , piece_id=246UC4PSMOZ2Q7557TLR4QUON7X2DW7SYQLH7534XSEDRKFK6YTA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167820397 (314h44m9.598900528s) [noise=true , address=77.90.31.17:4775 , piece_id=ZGZNQUHXPKQMWYHUPES6JLK74YGWSOWSRVPTOCKEUQQGWNQEG2DQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167897014 (314h44m9.598977135s) [noise=true , address=104.36.82.38:4713 , piece_id=RWBXRNQ2JMFJ4SYHSVGRJPUYUQMQN2URNCN6LJFW3PDVS6EVDY4Q, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168095995 (314h44m9.599176126s) [noise=true , address=104.36.82.5:4843 , piece_id=Z7WZ5UTYX32KJADY3FSEJGPAKCNG7LU5FEPXZ4TAXXOGFQ4EYBEA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168341745 (314h44m9.599421866s) [noise=true , address=45.91.116.12:4255 , piece_id=TNSZHSIHY7JDMMW6D3TL42AFSE2OPVVAK2NMK6C52VBWMHUHPQ7A, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168407420 (314h44m9.599487551s) [noise=true , address=79.127.201.215:4344 , piece_id=2JUCTHQ3BI3B3SE3AQFC63L3GNLYMYHTWLRRHLDXC3CCTHS5EGKA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168667458 (314h44m9.599747579s) [noise=true , address=45.91.116.10:4114 , piece_id=REEPZ6DH3SKWEHY2N67ZCVVCOPSEZEFT366BC4JVBI7GYJ4WKIGQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168907448 (314h44m9.599987569s) [noise=true , address=77.90.31.12:4594 , piece_id=WDHXZIEPQRHWZHTWJ6CGMIRNIQAI3NZ7YVP2IICUGBWS3SUB77FA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168965298 (314h44m9.600045429s) [noise=true , address=77.90.31.11:4559 , piece_id=6NWPHRGM3VPBZG7IV5755VGSMSG5KD2S6FQYS7RJZ63N52L7UBDQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169160421 (314h44m9.600240553s) [noise=true , address=104.36.82.34:4570 , piece_id=SG6AVXZ6JMY6BZMPO7YO4D2M4GSKMHFYQJKBHQFNXB2OWXX7PKXA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169288357 (314h44m9.600368488s) [noise=true , address=104.36.82.13:4212 , piece_id=YYDY7XTFA6EGFPWTPSFRIDSRJYTJLTKM54UIDQUYN5ZM7OVPCK5Q, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169361095 (314h44m9.600441216s) [noise=true , address=104.36.82.23:4477 , piece_id=LQWY3THAYVTLQQWDKSM445MVXVDF2DQCEDLBROM7DONFMXSSSAVA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169555206 (314h44m9.600635337s) [noise=true , address=45.91.116.7:4014 , piece_id=SPFLBQZCPK33TXQ6IBHFCB3WH7XO4PHREI5MWVPNCEQF4EYZP5RQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169563844 (314h44m9.600643975s) [noise=true , address=104.36.82.40:4963 , piece_id=6VIZW4MFPEOVQORZYUS2VOAA6Q4U66ZWZEHFXKILJTSRF2RQLNSA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169877724 (314h44m9.600957845s) [noise=true , address=104.36.82.9:4018 , piece_id=7S4G6XMWKXQ5U3FQDMW2VDLZYWFOBVRKMCFED6BCCIWE4QAA7I2A, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.170176706 (314h44m9.601256837s) [noise=true , address=104.36.82.2:4762 , piece_id=OIUTGZSMAVR2HYTRWWMWLSR5XTXJQHFXWA6D2WU5JDNBXLRYWCOA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:16.672535880 (314h44m11.103616011s) [upload=327443]: Next returned nil segment Aug 21 20:10:16.672555797 (314h44m11.103635928s) [upload=327443]: Waiting for error group managing segments... Aug 21 20:10:17.083344442 (314h44m11.514424563s) [noise=true , address=45.91.116.7:4014 , piece_id=SPFLBQZCPK33TXQ6IBHFCB3WH7XO4PHREI5MWVPNCEQF4EYZP5RQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.083887000 (314h44m11.514967132s) [noise=true , address=45.91.116.4:4962 , piece_id=7W2H6URV32LWUBDIDSMFL4HS6DRZ5GFQAU6G25Y3PNJHYDIVCPZA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.090288509 (314h44m11.52136868s) [noise=true , address=77.90.31.17:4785 , piece_id=XKOCSZKVZ63UDGWWXP6DO6FVAWHNUSZLQZ75AYL72JSFA3LND7PQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.141165807 (314h44m11.572245938s) [noise=true , address=104.36.82.9:4018 , piece_id=7S4G6XMWKXQ5U3FQDMW2VDLZYWFOBVRKMCFED6BCCIWE4QAA7I2A, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.157740550 (314h44m11.588820682s) [noise=true , address=45.91.116.12:4255 , piece_id=TNSZHSIHY7JDMMW6D3TL42AFSE2OPVVAK2NMK6C52VBWMHUHPQ7A, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.158401245 (314h44m11.589481376s) [noise=true , address=77.90.31.11:4559 , piece_id=6NWPHRGM3VPBZG7IV5755VGSMSG5KD2S6FQYS7RJZ63N52L7UBDQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.170813964 (314h44m11.601894085s) [noise=true , address=45.91.116.10:4114 , piece_id=REEPZ6DH3SKWEHY2N67ZCVVCOPSEZEFT366BC4JVBI7GYJ4WKIGQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.175584983 (314h44m11.606665114s) [noise=true , address=79.127.201.215:4344 , piece_id=2JUCTHQ3BI3B3SE3AQFC63L3GNLYMYHTWLRRHLDXC3CCTHS5EGKA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.197949946 (314h44m11.629030077s) [noise=true , address=77.90.31.12:4594 , piece_id=WDHXZIEPQRHWZHTWJ6CGMIRNIQAI3NZ7YVP2IICUGBWS3SUB77FA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.202226388 (314h44m11.633306509s) [noise=true , address=104.36.82.2:4762 , piece_id=OIUTGZSMAVR2HYTRWWMWLSR5XTXJQHFXWA6D2WU5JDNBXLRYWCOA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.202498088 (314h44m11.633578209s) [noise=true , address=104.36.82.11:4126 , piece_id=246UC4PSMOZ2Q7557TLR4QUON7X2DW7SYQLH7534XSEDRKFK6YTA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.204637111 (314h44m11.635717232s) [noise=true , address=104.36.82.34:4570 , piece_id=SG6AVXZ6JMY6BZMPO7YO4D2M4GSKMHFYQJKBHQFNXB2OWXX7PKXA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.208072624 (314h44m11.639152755s) [noise=true , address=104.36.82.40:4963 , piece_id=6VIZW4MFPEOVQORZYUS2VOAA6Q4U66ZWZEHFXKILJTSRF2RQLNSA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.251397182 (314h44m11.682477313s) [noise=true , address=104.36.82.38:4713 , piece_id=RWBXRNQ2JMFJ4SYHSVGRJPUYUQMQN2URNCN6LJFW3PDVS6EVDY4Q, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.251419764 (314h44m11.682499895s) [seg_pos={0 0}, upload=327443]: Segment reached optimal threshold of 14 pieces. Aug 21 20:10:17.251567658 (314h44m11.682647789s) [noise=true , address=77.90.31.17:4775 , piece_id=ZGZNQUHXPKQMWYHUPES6JLK74YGWSOWSRVPTOCKEUQQGWNQEG2DQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127VKvwi82kYfGZ6Wcv8BinaNUQBMQsgKLKNNLht5JPsBLN7ovt): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251639294 (314h44m11.682719425s) [noise=true , address=104.36.82.23:4477 , piece_id=LQWY3THAYVTLQQWDKSM445MVXVDF2DQCEDLBROM7DONFMXSSSAVA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RP2Zq643WWNtKEr34Tq1iZS29NYA2okr9Fg1jiCpaFTqfLauf): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251720610 (314h44m11.682800741s) [noise=true , address=104.36.82.5:4843 , piece_id=Z7WZ5UTYX32KJADY3FSEJGPAKCNG7LU5FEPXZ4TAXXOGFQ4EYBEA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HvL2JDP4GCsu3tziVEKRMh14nEzyRMRuWNSmVvs27LzHTbNpS): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251764264 (314h44m11.682844395s) [noise=true , address=104.36.82.13:4212 , piece_id=YYDY7XTFA6EGFPWTPSFRIDSRJYTJLTKM54UIDQUYN5ZM7OVPCK5Q, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1itAUtWeFERY4AKhPxE2CRwymp8qNC6wu6uhPt2n6aK58LRu24): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251780665 (314h44m11.682860796s) [seg_pos={0 0}, upload=327443]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 21 20:10:17.251866870 (314h44m11.682947001s) [upload=327443]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 21 20:10:17.251871469 (314h44m11.6829516s) [upload=327443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:10:19.201398003 (314h44m13.632478134s) [upload=327443]: Done uploading segments... Aug 21 20:10:19.201416889 (314h44m13.63249702s) [upload=327443]: Upload finished. err: <nil> Duration: 5.823293847s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.805013718s FAIL (2025-08-21 20:10:35 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.805065598s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X.................X...................................X] duration: 5.601912545s FAIL (2025-08-21 20:10:19 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 21 20:10:14.716582768 (314h44m9.147662899s) [upload=327443]: Starting upload Aug 21 20:10:14.716649245 (314h44m9.147729366s) [upload=327443]: Done starting upload Aug 21 20:10:14.716772782 (314h44m9.147852913s) [upload=327443]: Uploading segments... Aug 21 20:10:14.716914553 (314h44m9.147994684s) [upload=327443]: Got next segment. Inline: false Aug 21 20:10:14.716961734 (314h44m9.148041855s) [upload=327443]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:10:14.716966182 (314h44m9.148046313s) [upload=327443]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 21 20:10:15.167114827 (314h44m9.598194958s) [seg_pos={0 0}, upload=327443]: Begin upload segment... Aug 21 20:10:15.167157349 (314h44m9.59823748s) [seg_pos={0 0}, upload=327443]: Done begin upload segment. Aug 21 20:10:15.167177778 (314h44m9.598257899s) [noise=true , address=77.90.31.17:4785 , piece_id=XKOCSZKVZ63UDGWWXP6DO6FVAWHNUSZLQZ75AYL72JSFA3LND7PQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167269874 (314h44m9.598349995s) [noise=true , address=45.91.116.4:4962 , piece_id=7W2H6URV32LWUBDIDSMFL4HS6DRZ5GFQAU6G25Y3PNJHYDIVCPZA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167549559 (314h44m9.59862969s) [noise=true , address=104.36.82.11:4126 , piece_id=246UC4PSMOZ2Q7557TLR4QUON7X2DW7SYQLH7534XSEDRKFK6YTA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167820397 (314h44m9.598900528s) [noise=true , address=77.90.31.17:4775 , piece_id=ZGZNQUHXPKQMWYHUPES6JLK74YGWSOWSRVPTOCKEUQQGWNQEG2DQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.167897014 (314h44m9.598977135s) [noise=true , address=104.36.82.38:4713 , piece_id=RWBXRNQ2JMFJ4SYHSVGRJPUYUQMQN2URNCN6LJFW3PDVS6EVDY4Q, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168095995 (314h44m9.599176126s) [noise=true , address=104.36.82.5:4843 , piece_id=Z7WZ5UTYX32KJADY3FSEJGPAKCNG7LU5FEPXZ4TAXXOGFQ4EYBEA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168341745 (314h44m9.599421866s) [noise=true , address=45.91.116.12:4255 , piece_id=TNSZHSIHY7JDMMW6D3TL42AFSE2OPVVAK2NMK6C52VBWMHUHPQ7A, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168407420 (314h44m9.599487551s) [noise=true , address=79.127.201.215:4344 , piece_id=2JUCTHQ3BI3B3SE3AQFC63L3GNLYMYHTWLRRHLDXC3CCTHS5EGKA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168667458 (314h44m9.599747579s) [noise=true , address=45.91.116.10:4114 , piece_id=REEPZ6DH3SKWEHY2N67ZCVVCOPSEZEFT366BC4JVBI7GYJ4WKIGQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168907448 (314h44m9.599987569s) [noise=true , address=77.90.31.12:4594 , piece_id=WDHXZIEPQRHWZHTWJ6CGMIRNIQAI3NZ7YVP2IICUGBWS3SUB77FA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.168965298 (314h44m9.600045429s) [noise=true , address=77.90.31.11:4559 , piece_id=6NWPHRGM3VPBZG7IV5755VGSMSG5KD2S6FQYS7RJZ63N52L7UBDQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169160421 (314h44m9.600240553s) [noise=true , address=104.36.82.34:4570 , piece_id=SG6AVXZ6JMY6BZMPO7YO4D2M4GSKMHFYQJKBHQFNXB2OWXX7PKXA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169288357 (314h44m9.600368488s) [noise=true , address=104.36.82.13:4212 , piece_id=YYDY7XTFA6EGFPWTPSFRIDSRJYTJLTKM54UIDQUYN5ZM7OVPCK5Q, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169361095 (314h44m9.600441216s) [noise=true , address=104.36.82.23:4477 , piece_id=LQWY3THAYVTLQQWDKSM445MVXVDF2DQCEDLBROM7DONFMXSSSAVA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169555206 (314h44m9.600635337s) [noise=true , address=45.91.116.7:4014 , piece_id=SPFLBQZCPK33TXQ6IBHFCB3WH7XO4PHREI5MWVPNCEQF4EYZP5RQ, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169563844 (314h44m9.600643975s) [noise=true , address=104.36.82.40:4963 , piece_id=6VIZW4MFPEOVQORZYUS2VOAA6Q4U66ZWZEHFXKILJTSRF2RQLNSA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.169877724 (314h44m9.600957845s) [noise=true , address=104.36.82.9:4018 , piece_id=7S4G6XMWKXQ5U3FQDMW2VDLZYWFOBVRKMCFED6BCCIWE4QAA7I2A, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:15.170176706 (314h44m9.601256837s) [noise=true , address=104.36.82.2:4762 , piece_id=OIUTGZSMAVR2HYTRWWMWLSR5XTXJQHFXWA6D2WU5JDNBXLRYWCOA, seg_pos={0 0}, upload=327443]: Uploading piece... Aug 21 20:10:16.672535880 (314h44m11.103616011s) [upload=327443]: Next returned nil segment Aug 21 20:10:16.672555797 (314h44m11.103635928s) [upload=327443]: Waiting for error group managing segments... Aug 21 20:10:17.083344442 (314h44m11.514424563s) [noise=true , address=45.91.116.7:4014 , piece_id=SPFLBQZCPK33TXQ6IBHFCB3WH7XO4PHREI5MWVPNCEQF4EYZP5RQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.083887000 (314h44m11.514967132s) [noise=true , address=45.91.116.4:4962 , piece_id=7W2H6URV32LWUBDIDSMFL4HS6DRZ5GFQAU6G25Y3PNJHYDIVCPZA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.090288509 (314h44m11.52136868s) [noise=true , address=77.90.31.17:4785 , piece_id=XKOCSZKVZ63UDGWWXP6DO6FVAWHNUSZLQZ75AYL72JSFA3LND7PQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.141165807 (314h44m11.572245938s) [noise=true , address=104.36.82.9:4018 , piece_id=7S4G6XMWKXQ5U3FQDMW2VDLZYWFOBVRKMCFED6BCCIWE4QAA7I2A, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.157740550 (314h44m11.588820682s) [noise=true , address=45.91.116.12:4255 , piece_id=TNSZHSIHY7JDMMW6D3TL42AFSE2OPVVAK2NMK6C52VBWMHUHPQ7A, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.158401245 (314h44m11.589481376s) [noise=true , address=77.90.31.11:4559 , piece_id=6NWPHRGM3VPBZG7IV5755VGSMSG5KD2S6FQYS7RJZ63N52L7UBDQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.170813964 (314h44m11.601894085s) [noise=true , address=45.91.116.10:4114 , piece_id=REEPZ6DH3SKWEHY2N67ZCVVCOPSEZEFT366BC4JVBI7GYJ4WKIGQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.175584983 (314h44m11.606665114s) [noise=true , address=79.127.201.215:4344 , piece_id=2JUCTHQ3BI3B3SE3AQFC63L3GNLYMYHTWLRRHLDXC3CCTHS5EGKA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.197949946 (314h44m11.629030077s) [noise=true , address=77.90.31.12:4594 , piece_id=WDHXZIEPQRHWZHTWJ6CGMIRNIQAI3NZ7YVP2IICUGBWS3SUB77FA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.202226388 (314h44m11.633306509s) [noise=true , address=104.36.82.2:4762 , piece_id=OIUTGZSMAVR2HYTRWWMWLSR5XTXJQHFXWA6D2WU5JDNBXLRYWCOA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.202498088 (314h44m11.633578209s) [noise=true , address=104.36.82.11:4126 , piece_id=246UC4PSMOZ2Q7557TLR4QUON7X2DW7SYQLH7534XSEDRKFK6YTA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.204637111 (314h44m11.635717232s) [noise=true , address=104.36.82.34:4570 , piece_id=SG6AVXZ6JMY6BZMPO7YO4D2M4GSKMHFYQJKBHQFNXB2OWXX7PKXA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.208072624 (314h44m11.639152755s) [noise=true , address=104.36.82.40:4963 , piece_id=6VIZW4MFPEOVQORZYUS2VOAA6Q4U66ZWZEHFXKILJTSRF2RQLNSA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.251397182 (314h44m11.682477313s) [noise=true , address=104.36.82.38:4713 , piece_id=RWBXRNQ2JMFJ4SYHSVGRJPUYUQMQN2URNCN6LJFW3PDVS6EVDY4Q, seg_pos={0 0}, upload=327443]: Done uploading piece. err: <nil> Aug 21 20:10:17.251419764 (314h44m11.682499895s) [seg_pos={0 0}, upload=327443]: Segment reached optimal threshold of 14 pieces. Aug 21 20:10:17.251567658 (314h44m11.682647789s) [noise=true , address=77.90.31.17:4775 , piece_id=ZGZNQUHXPKQMWYHUPES6JLK74YGWSOWSRVPTOCKEUQQGWNQEG2DQ, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127VKvwi82kYfGZ6Wcv8BinaNUQBMQsgKLKNNLht5JPsBLN7ovt): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251639294 (314h44m11.682719425s) [noise=true , address=104.36.82.23:4477 , piece_id=LQWY3THAYVTLQQWDKSM445MVXVDF2DQCEDLBROM7DONFMXSSSAVA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RP2Zq643WWNtKEr34Tq1iZS29NYA2okr9Fg1jiCpaFTqfLauf): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251720610 (314h44m11.682800741s) [noise=true , address=104.36.82.5:4843 , piece_id=Z7WZ5UTYX32KJADY3FSEJGPAKCNG7LU5FEPXZ4TAXXOGFQ4EYBEA, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HvL2JDP4GCsu3tziVEKRMh14nEzyRMRuWNSmVvs27LzHTbNpS): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251764264 (314h44m11.682844395s) [noise=true , address=104.36.82.13:4212 , piece_id=YYDY7XTFA6EGFPWTPSFRIDSRJYTJLTKM54UIDQUYN5ZM7OVPCK5Q, seg_pos={0 0}, upload=327443]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1itAUtWeFERY4AKhPxE2CRwymp8qNC6wu6uhPt2n6aK58LRu24): protocol: expected piece hash; context canceled; EOF Aug 21 20:10:17.251780665 (314h44m11.682860796s) [seg_pos={0 0}, upload=327443]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 21 20:10:17.251866870 (314h44m11.682947001s) [upload=327443]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 21 20:10:17.251871469 (314h44m11.6829516s) [upload=327443]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:10:19.201398003 (314h44m13.632478134s) [upload=327443]: Done uploading segments... Aug 21 20:10:19.201416889 (314h44m13.63249702s) [upload=327443]: Upload finished. err: <nil> Duration: 5.601950387s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.X..................X..............................X........] duration: 1.880790026s FAIL (2025-08-21 20:10:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 20:05:21.070335328 (314h39m15.50141547s) [upload=327359]: Starting upload Aug 21 20:05:21.070433486 (314h39m15.501513657s) [upload=327359]: Done starting upload Aug 21 20:05:21.070504632 (314h39m15.501584773s) [upload=327359]: Uploading segments... Aug 21 20:05:21.070680819 (314h39m15.501761s) [upload=327359]: Got next segment. Inline: true Aug 21 20:05:21.070687241 (314h39m15.501767372s) [upload=327359]: Waiting for error group managing segments... Aug 21 20:05:21.070740553 (314h39m15.501820694s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:05:21.070747186 (314h39m15.501827338s) [upload=327359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 21 20:05:21.070754069 (314h39m15.501834221s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:05:21.283591166 (314h39m15.714671297s) [upload=327359]: Done uploading segments... Aug 21 20:05:21.283608600 (314h39m15.714688731s) [upload=327359]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.880825966s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..X..................X..............................X.......] duration: 2.138680258s FAIL (2025-08-21 20:09:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 20:05:21.070335328 (314h39m15.50141547s) [upload=327359]: Starting upload Aug 21 20:05:21.070433486 (314h39m15.501513657s) [upload=327359]: Done starting upload Aug 21 20:05:21.070504632 (314h39m15.501584773s) [upload=327359]: Uploading segments... Aug 21 20:05:21.070680819 (314h39m15.501761s) [upload=327359]: Got next segment. Inline: true Aug 21 20:05:21.070687241 (314h39m15.501767372s) [upload=327359]: Waiting for error group managing segments... Aug 21 20:05:21.070740553 (314h39m15.501820694s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:05:21.070747186 (314h39m15.501827338s) [upload=327359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 21 20:05:21.070754069 (314h39m15.501834221s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:05:21.283591166 (314h39m15.714671297s) [upload=327359]: Done uploading segments... Aug 21 20:05:21.283608600 (314h39m15.714688731s) [upload=327359]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.138705487s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.756033107s FAIL (2025-08-21 20:09:33 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.756062101s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...X..................X..............................X......] duration: 2.102367286s FAIL (2025-08-21 20:09:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 20:05:21.070335328 (314h39m15.50141547s) [upload=327359]: Starting upload Aug 21 20:05:21.070433486 (314h39m15.501513657s) [upload=327359]: Done starting upload Aug 21 20:05:21.070504632 (314h39m15.501584773s) [upload=327359]: Uploading segments... Aug 21 20:05:21.070680819 (314h39m15.501761s) [upload=327359]: Got next segment. Inline: true Aug 21 20:05:21.070687241 (314h39m15.501767372s) [upload=327359]: Waiting for error group managing segments... Aug 21 20:05:21.070740553 (314h39m15.501820694s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:05:21.070747186 (314h39m15.501827338s) [upload=327359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 21 20:05:21.070754069 (314h39m15.501834221s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:05:21.283591166 (314h39m15.714671297s) [upload=327359]: Done uploading segments... Aug 21 20:05:21.283608600 (314h39m15.714688731s) [upload=327359]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.102403976s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X.................X...................................] duration: 5.940588992s FAIL (2025-08-21 20:09:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:43:22.884337203 (314h17m17.315417324s) [upload=326975]: Starting upload Aug 21 19:43:22.884393010 (314h17m17.315473131s) [upload=326975]: Done starting upload Aug 21 19:43:22.884456912 (314h17m17.315537043s) [upload=326975]: Uploading segments... Aug 21 19:43:22.884623320 (314h17m17.315703451s) [upload=326975]: Got next segment. Inline: false Aug 21 19:43:22.884653899 (314h17m17.31573403s) [upload=326975]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 19:43:22.884658407 (314h17m17.315738538s) [upload=326975]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 21 19:43:23.102820154 (314h17m17.533900295s) [upload=326975]: Next segment err: metaclient: Too Many Requests Aug 21 19:43:23.102842577 (314h17m17.533922708s) [upload=326975]: Done uploading segments... Aug 21 19:43:23.102856203 (314h17m17.533936334s) [upload=326975]: Upload finished. err: metaclient: Too Many Requests Duration: 5.940616094s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X.................X..................................] duration: 5.872519479s FAIL (2025-08-21 20:08:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:43:22.884337203 (314h17m17.315417324s) [upload=326975]: Starting upload Aug 21 19:43:22.884393010 (314h17m17.315473131s) [upload=326975]: Done starting upload Aug 21 19:43:22.884456912 (314h17m17.315537043s) [upload=326975]: Uploading segments... Aug 21 19:43:22.884623320 (314h17m17.315703451s) [upload=326975]: Got next segment. Inline: false Aug 21 19:43:22.884653899 (314h17m17.31573403s) [upload=326975]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 19:43:22.884658407 (314h17m17.315738538s) [upload=326975]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 21 19:43:23.102820154 (314h17m17.533900295s) [upload=326975]: Next segment err: metaclient: Too Many Requests Aug 21 19:43:23.102842577 (314h17m17.533922708s) [upload=326975]: Done uploading segments... Aug 21 19:43:23.102856203 (314h17m17.533936334s) [upload=326975]: Upload finished. err: metaclient: Too Many Requests Duration: 5.872543776s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [....X..................X..............................X.....] duration: 1.959536529s FAIL (2025-08-21 20:08:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 20:05:21.070335328 (314h39m15.50141547s) [upload=327359]: Starting upload Aug 21 20:05:21.070433486 (314h39m15.501513657s) [upload=327359]: Done starting upload Aug 21 20:05:21.070504632 (314h39m15.501584773s) [upload=327359]: Uploading segments... Aug 21 20:05:21.070680819 (314h39m15.501761s) [upload=327359]: Got next segment. Inline: true Aug 21 20:05:21.070687241 (314h39m15.501767372s) [upload=327359]: Waiting for error group managing segments... Aug 21 20:05:21.070740553 (314h39m15.501820694s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:05:21.070747186 (314h39m15.501827338s) [upload=327359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 21 20:05:21.070754069 (314h39m15.501834221s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:05:21.283591166 (314h39m15.714671297s) [upload=327359]: Done uploading segments... Aug 21 20:05:21.283608600 (314h39m15.714688731s) [upload=327359]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.95956917s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.015476251s FAIL (2025-08-21 20:08:23 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.015492942s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.064484189s FAIL (2025-08-21 20:07:59 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.064510961s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [X.......X.................X.................................] duration: 6.025517703s FAIL (2025-08-21 20:07:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:43:22.884337203 (314h17m17.315417324s) [upload=326975]: Starting upload Aug 21 19:43:22.884393010 (314h17m17.315473131s) [upload=326975]: Done starting upload Aug 21 19:43:22.884456912 (314h17m17.315537043s) [upload=326975]: Uploading segments... Aug 21 19:43:22.884623320 (314h17m17.315703451s) [upload=326975]: Got next segment. Inline: false Aug 21 19:43:22.884653899 (314h17m17.31573403s) [upload=326975]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 19:43:22.884658407 (314h17m17.315738538s) [upload=326975]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 21 19:43:23.102820154 (314h17m17.533900295s) [upload=326975]: Next segment err: metaclient: Too Many Requests Aug 21 19:43:23.102842577 (314h17m17.533922708s) [upload=326975]: Done uploading segments... Aug 21 19:43:23.102856203 (314h17m17.533936334s) [upload=326975]: Upload finished. err: metaclient: Too Many Requests Duration: 6.025544273s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.....X..................X..............................X....] duration: 1.996209226s FAIL (2025-08-21 20:07:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 20:05:21.070335328 (314h39m15.50141547s) [upload=327359]: Starting upload Aug 21 20:05:21.070433486 (314h39m15.501513657s) [upload=327359]: Done starting upload Aug 21 20:05:21.070504632 (314h39m15.501584773s) [upload=327359]: Uploading segments... Aug 21 20:05:21.070680819 (314h39m15.501761s) [upload=327359]: Got next segment. Inline: true Aug 21 20:05:21.070687241 (314h39m15.501767372s) [upload=327359]: Waiting for error group managing segments... Aug 21 20:05:21.070740553 (314h39m15.501820694s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:05:21.070747186 (314h39m15.501827338s) [upload=327359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 21 20:05:21.070754069 (314h39m15.501834221s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:05:21.283591166 (314h39m15.714671297s) [upload=327359]: Done uploading segments... Aug 21 20:05:21.283608600 (314h39m15.714688731s) [upload=327359]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.99639973s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X.......X.................X................................] duration: 5.850152042s FAIL (2025-08-21 20:07:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:43:22.884337203 (314h17m17.315417324s) [upload=326975]: Starting upload Aug 21 19:43:22.884393010 (314h17m17.315473131s) [upload=326975]: Done starting upload Aug 21 19:43:22.884456912 (314h17m17.315537043s) [upload=326975]: Uploading segments... Aug 21 19:43:22.884623320 (314h17m17.315703451s) [upload=326975]: Got next segment. Inline: false Aug 21 19:43:22.884653899 (314h17m17.31573403s) [upload=326975]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 19:43:22.884658407 (314h17m17.315738538s) [upload=326975]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 21 19:43:23.102820154 (314h17m17.533900295s) [upload=326975]: Next segment err: metaclient: Too Many Requests Aug 21 19:43:23.102842577 (314h17m17.533922708s) [upload=326975]: Done uploading segments... Aug 21 19:43:23.102856203 (314h17m17.533936334s) [upload=326975]: Upload finished. err: metaclient: Too Many Requests Duration: 5.850185725s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [......X..................X..............................X...] duration: 1.858365656s FAIL (2025-08-21 20:07:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 20:05:21.070335328 (314h39m15.50141547s) [upload=327359]: Starting upload Aug 21 20:05:21.070433486 (314h39m15.501513657s) [upload=327359]: Done starting upload Aug 21 20:05:21.070504632 (314h39m15.501584773s) [upload=327359]: Uploading segments... Aug 21 20:05:21.070680819 (314h39m15.501761s) [upload=327359]: Got next segment. Inline: true Aug 21 20:05:21.070687241 (314h39m15.501767372s) [upload=327359]: Waiting for error group managing segments... Aug 21 20:05:21.070740553 (314h39m15.501820694s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:05:21.070747186 (314h39m15.501827338s) [upload=327359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 21 20:05:21.070754069 (314h39m15.501834221s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:05:21.283591166 (314h39m15.714671297s) [upload=327359]: Done uploading segments... Aug 21 20:05:21.283608600 (314h39m15.714688731s) [upload=327359]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.858396565s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.......X..................X..............................X..] duration: 2.199102324s FAIL (2025-08-21 20:06:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 20:05:21.070335328 (314h39m15.50141547s) [upload=327359]: Starting upload Aug 21 20:05:21.070433486 (314h39m15.501513657s) [upload=327359]: Done starting upload Aug 21 20:05:21.070504632 (314h39m15.501584773s) [upload=327359]: Uploading segments... Aug 21 20:05:21.070680819 (314h39m15.501761s) [upload=327359]: Got next segment. Inline: true Aug 21 20:05:21.070687241 (314h39m15.501767372s) [upload=327359]: Waiting for error group managing segments... Aug 21 20:05:21.070740553 (314h39m15.501820694s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:05:21.070747186 (314h39m15.501827338s) [upload=327359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 21 20:05:21.070754069 (314h39m15.501834221s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:05:21.283591166 (314h39m15.714671297s) [upload=327359]: Done uploading segments... Aug 21 20:05:21.283608600 (314h39m15.714688731s) [upload=327359]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.199128815s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 5.965389937s FAIL (2025-08-21 20:06:57 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.965418651s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 6.016405096s FAIL (2025-08-21 20:06:35 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.01643297s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [X.X.......X.................X...............................] duration: 5.89392664s FAIL (2025-08-21 20:06:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:43:22.884337203 (314h17m17.315417324s) [upload=326975]: Starting upload Aug 21 19:43:22.884393010 (314h17m17.315473131s) [upload=326975]: Done starting upload Aug 21 19:43:22.884456912 (314h17m17.315537043s) [upload=326975]: Uploading segments... Aug 21 19:43:22.884623320 (314h17m17.315703451s) [upload=326975]: Got next segment. Inline: false Aug 21 19:43:22.884653899 (314h17m17.31573403s) [upload=326975]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 19:43:22.884658407 (314h17m17.315738538s) [upload=326975]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 21 19:43:23.102820154 (314h17m17.533900295s) [upload=326975]: Next segment err: metaclient: Too Many Requests Aug 21 19:43:23.102842577 (314h17m17.533922708s) [upload=326975]: Done uploading segments... Aug 21 19:43:23.102856203 (314h17m17.533936334s) [upload=326975]: Upload finished. err: metaclient: Too Many Requests Duration: 5.893956248s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [........X..................X..............................X.] duration: 2.126543743s FAIL (2025-08-21 20:06:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 20:05:21.070335328 (314h39m15.50141547s) [upload=327359]: Starting upload Aug 21 20:05:21.070433486 (314h39m15.501513657s) [upload=327359]: Done starting upload Aug 21 20:05:21.070504632 (314h39m15.501584773s) [upload=327359]: Uploading segments... Aug 21 20:05:21.070680819 (314h39m15.501761s) [upload=327359]: Got next segment. Inline: true Aug 21 20:05:21.070687241 (314h39m15.501767372s) [upload=327359]: Waiting for error group managing segments... Aug 21 20:05:21.070740553 (314h39m15.501820694s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 20:05:21.070747186 (314h39m15.501827338s) [upload=327359]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 21 20:05:21.070754069 (314h39m15.501834221s) [upload=327359]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 21 20:05:21.283591166 (314h39m15.714671297s) [upload=327359]: Done uploading segments... Aug 21 20:05:21.283608600 (314h39m15.714688731s) [upload=327359]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.126692017s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.767582083s FAIL (2025-08-21 20:06:07 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.767611579s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.X.X.......X.................X..............................] duration: 5.896676709s FAIL (2025-08-21 20:06:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:43:22.884337203 (314h17m17.315417324s) [upload=326975]: Starting upload Aug 21 19:43:22.884393010 (314h17m17.315473131s) [upload=326975]: Done starting upload Aug 21 19:43:22.884456912 (314h17m17.315537043s) [upload=326975]: Uploading segments... Aug 21 19:43:22.884623320 (314h17m17.315703451s) [upload=326975]: Got next segment. Inline: false Aug 21 19:43:22.884653899 (314h17m17.31573403s) [upload=326975]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 19:43:22.884658407 (314h17m17.315738538s) [upload=326975]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 21 19:43:23.102820154 (314h17m17.533900295s) [upload=326975]: Next segment err: metaclient: Too Many Requests Aug 21 19:43:23.102842577 (314h17m17.533922708s) [upload=326975]: Done uploading segments... Aug 21 19:43:23.102856203 (314h17m17.533936334s) [upload=326975]: Upload finished. err: metaclient: Too Many Requests Duration: 5.896706285s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..X.X.......X.................X.............................] duration: 5.829878038s FAIL (2025-08-21 20:05:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:43:22.884337203 (314h17m17.315417324s) [upload=326975]: Starting upload Aug 21 19:43:22.884393010 (314h17m17.315473131s) [upload=326975]: Done starting upload Aug 21 19:43:22.884456912 (314h17m17.315537043s) [upload=326975]: Uploading segments... Aug 21 19:43:22.884623320 (314h17m17.315703451s) [upload=326975]: Got next segment. Inline: false Aug 21 19:43:22.884653899 (314h17m17.31573403s) [upload=326975]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 21 19:43:22.884658407 (314h17m17.315738538s) [upload=326975]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 21 19:43:23.102820154 (314h17m17.533900295s) [upload=326975]: Next segment err: metaclient: Too Many Requests Aug 21 19:43:23.102842577 (314h17m17.533922708s) [upload=326975]: Done uploading segments... Aug 21 19:43:23.102856203 (314h17m17.533936334s) [upload=326975]: Upload finished. err: metaclient: Too Many Requests Duration: 5.829903316s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)