Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.918860346s FAIL (2025-08-06 23:00:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== VGIBWSTNBG3T5ZAPE5AQ, seg_pos={0 0}, upload=7839]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: context canceled; context canceled Aug 6 22:22:24.227847111 (8h13m32.16626296s) [seg_pos={0 0}, upload=7839]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:22:24.227953294 (8h13m32.166369183s) [upload=7839]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:22:24.227978793 (8h13m32.166394642s) [upload=7839]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:22:24.511796264 (8h13m32.450212103s) [upload=7839]: Done uploading segments... Aug 6 22:22:24.511838736 (8h13m32.450254585s) [upload=7839]: Upload finished. err: <nil> Aug 6 22:23:04.224450786 (8h14m12.162866635s) [upload=7849]: Starting upload Aug 6 22:23:04.224529026 (8h14m12.162944865s) [upload=7849]: Done starting upload Aug 6 22:23:04.224618757 (8h14m12.163034606s) [upload=7849]: Uploading segments... Aug 6 22:23:04.224730752 (8h14m12.163146591s) [upload=7849]: Got next segment. Inline: false Aug 6 22:23:04.224756230 (8h14m12.163172079s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 6 22:23:04.224760679 (8h14m12.163176528s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 6 22:23:04.456921276 (8h14m12.395337125s) [seg_pos={0 0}, upload=7849]: Begin upload segment... Aug 6 22:23:04.456981781 (8h14m12.395397631s) [seg_pos={0 0}, upload=7849]: Done begin upload segment. Aug 6 22:23:04.457004415 (8h14m12.395420264s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457046024 (8h14m12.395461873s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457558906 (8h14m12.395974755s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457926018 (8h14m12.396341867s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458231853 (8h14m12.396647702s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458325132 (8h14m12.396740981s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458575431 (8h14m12.39699128s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458624104 (8h14m12.397039953s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458906444 (8h14m12.397322293s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458920901 (8h14m12.39733676s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459170670 (8h14m12.397586519s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459279838 (8h14m12.397695687s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459426770 (8h14m12.397842619s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459604800 (8h14m12.398020639s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459897921 (8h14m12.39831377s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.460116189 (8h14m12.398532038s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:06.263153507 (8h14m14.201569346s) [upload=7849]: Next returned nil segment Aug 6 22:23:06.263169687 (8h14m14.201585536s) [upload=7849]: Waiting for error group managing segments... Aug 6 22:23:06.593041172 (8h14m14.531457011s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.614464201 (8h14m14.55288005s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.646281806 (8h14m14.584697655s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.684232226 (8h14m14.622648075s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.736100365 (8h14m14.674516214s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.744475628 (8h14m14.682891477s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.775567917 (8h14m14.713983766s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.785843175 (8h14m14.724259025s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.793127933 (8h14m14.731543773s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.805200522 (8h14m14.743616371s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.840123841 (8h14m14.77853969s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.895304316 (8h14m14.833720165s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915155747 (8h14m14.853571596s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915179071 (8h14m14.85359492s) [seg_pos={0 0}, upload=7849]: Segment reached optimal threshold of 13 pieces. Aug 6 22:23:06.915296927 (8h14m14.853712816s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 6 22:23:06.915493824 (8h14m14.853909673s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection Aug 6 22:23:06.915533600 (8h14m14.853949449s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: read tcp 188.166.225.170:33788->104.36.82.3:4767: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 6 22:23:06.915551634 (8h14m14.853967483s) [seg_pos={0 0}, upload=7849]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:23:06.915610066 (8h14m14.854025915s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:23:06.915613933 (8h14m14.854029772s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:23:07.212500018 (8h14m15.150915887s) [upload=7849]: Done uploading segments... Aug 6 22:23:07.212522912 (8h14m15.150938751s) [upload=7849]: Upload finished. err: <nil> Duration: 6.918895935s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 6.288039271s FAIL (2025-08-06 23:00:02 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/4M") Duration: 6.288065111s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.................X.............X.......X....................] duration: 1.927780166s FAIL (2025-08-06 22:59:56 UTC) orbiter warning: operation: Commit uplink: too many requests Duration: 1.927799402s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.328859391s FAIL (2025-08-06 22:59:39 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/4M") Duration: 6.32888547s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.173211355s FAIL (2025-08-06 22:59:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== VGIBWSTNBG3T5ZAPE5AQ, seg_pos={0 0}, upload=7839]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: context canceled; context canceled Aug 6 22:22:24.227847111 (8h13m32.16626296s) [seg_pos={0 0}, upload=7839]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:22:24.227953294 (8h13m32.166369183s) [upload=7839]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:22:24.227978793 (8h13m32.166394642s) [upload=7839]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:22:24.511796264 (8h13m32.450212103s) [upload=7839]: Done uploading segments... Aug 6 22:22:24.511838736 (8h13m32.450254585s) [upload=7839]: Upload finished. err: <nil> Aug 6 22:23:04.224450786 (8h14m12.162866635s) [upload=7849]: Starting upload Aug 6 22:23:04.224529026 (8h14m12.162944865s) [upload=7849]: Done starting upload Aug 6 22:23:04.224618757 (8h14m12.163034606s) [upload=7849]: Uploading segments... Aug 6 22:23:04.224730752 (8h14m12.163146591s) [upload=7849]: Got next segment. Inline: false Aug 6 22:23:04.224756230 (8h14m12.163172079s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 6 22:23:04.224760679 (8h14m12.163176528s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 6 22:23:04.456921276 (8h14m12.395337125s) [seg_pos={0 0}, upload=7849]: Begin upload segment... Aug 6 22:23:04.456981781 (8h14m12.395397631s) [seg_pos={0 0}, upload=7849]: Done begin upload segment. Aug 6 22:23:04.457004415 (8h14m12.395420264s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457046024 (8h14m12.395461873s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457558906 (8h14m12.395974755s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457926018 (8h14m12.396341867s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458231853 (8h14m12.396647702s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458325132 (8h14m12.396740981s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458575431 (8h14m12.39699128s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458624104 (8h14m12.397039953s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458906444 (8h14m12.397322293s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458920901 (8h14m12.39733676s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459170670 (8h14m12.397586519s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459279838 (8h14m12.397695687s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459426770 (8h14m12.397842619s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459604800 (8h14m12.398020639s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459897921 (8h14m12.39831377s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.460116189 (8h14m12.398532038s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:06.263153507 (8h14m14.201569346s) [upload=7849]: Next returned nil segment Aug 6 22:23:06.263169687 (8h14m14.201585536s) [upload=7849]: Waiting for error group managing segments... Aug 6 22:23:06.593041172 (8h14m14.531457011s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.614464201 (8h14m14.55288005s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.646281806 (8h14m14.584697655s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.684232226 (8h14m14.622648075s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.736100365 (8h14m14.674516214s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.744475628 (8h14m14.682891477s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.775567917 (8h14m14.713983766s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.785843175 (8h14m14.724259025s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.793127933 (8h14m14.731543773s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.805200522 (8h14m14.743616371s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.840123841 (8h14m14.77853969s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.895304316 (8h14m14.833720165s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915155747 (8h14m14.853571596s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915179071 (8h14m14.85359492s) [seg_pos={0 0}, upload=7849]: Segment reached optimal threshold of 13 pieces. Aug 6 22:23:06.915296927 (8h14m14.853712816s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 6 22:23:06.915493824 (8h14m14.853909673s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection Aug 6 22:23:06.915533600 (8h14m14.853949449s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: read tcp 188.166.225.170:33788->104.36.82.3:4767: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 6 22:23:06.915551634 (8h14m14.853967483s) [seg_pos={0 0}, upload=7849]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:23:06.915610066 (8h14m14.854025915s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:23:06.915613933 (8h14m14.854029772s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:23:07.212500018 (8h14m15.150915887s) [upload=7849]: Done uploading segments... Aug 6 22:23:07.212522912 (8h14m15.150938751s) [upload=7849]: Upload finished. err: <nil> Duration: 6.173235373s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 8.622689834s FAIL (2025-08-06 22:59:18 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/4M") Duration: 8.622717438s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..................X.............X.......X...................] duration: 1.872367496s FAIL (2025-08-06 22:58:51 UTC) orbiter warning: operation: Commit uplink: too many requests Duration: 1.872390329s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.699778747s FAIL (2025-08-06 22:58:43 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/4M") Duration: 5.699803545s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.913329143s FAIL (2025-08-06 22:58:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== VGIBWSTNBG3T5ZAPE5AQ, seg_pos={0 0}, upload=7839]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: context canceled; context canceled Aug 6 22:22:24.227847111 (8h13m32.16626296s) [seg_pos={0 0}, upload=7839]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:22:24.227953294 (8h13m32.166369183s) [upload=7839]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:22:24.227978793 (8h13m32.166394642s) [upload=7839]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:22:24.511796264 (8h13m32.450212103s) [upload=7839]: Done uploading segments... Aug 6 22:22:24.511838736 (8h13m32.450254585s) [upload=7839]: Upload finished. err: <nil> Aug 6 22:23:04.224450786 (8h14m12.162866635s) [upload=7849]: Starting upload Aug 6 22:23:04.224529026 (8h14m12.162944865s) [upload=7849]: Done starting upload Aug 6 22:23:04.224618757 (8h14m12.163034606s) [upload=7849]: Uploading segments... Aug 6 22:23:04.224730752 (8h14m12.163146591s) [upload=7849]: Got next segment. Inline: false Aug 6 22:23:04.224756230 (8h14m12.163172079s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 6 22:23:04.224760679 (8h14m12.163176528s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 6 22:23:04.456921276 (8h14m12.395337125s) [seg_pos={0 0}, upload=7849]: Begin upload segment... Aug 6 22:23:04.456981781 (8h14m12.395397631s) [seg_pos={0 0}, upload=7849]: Done begin upload segment. Aug 6 22:23:04.457004415 (8h14m12.395420264s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457046024 (8h14m12.395461873s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457558906 (8h14m12.395974755s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457926018 (8h14m12.396341867s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458231853 (8h14m12.396647702s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458325132 (8h14m12.396740981s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458575431 (8h14m12.39699128s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458624104 (8h14m12.397039953s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458906444 (8h14m12.397322293s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458920901 (8h14m12.39733676s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459170670 (8h14m12.397586519s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459279838 (8h14m12.397695687s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459426770 (8h14m12.397842619s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459604800 (8h14m12.398020639s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459897921 (8h14m12.39831377s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.460116189 (8h14m12.398532038s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:06.263153507 (8h14m14.201569346s) [upload=7849]: Next returned nil segment Aug 6 22:23:06.263169687 (8h14m14.201585536s) [upload=7849]: Waiting for error group managing segments... Aug 6 22:23:06.593041172 (8h14m14.531457011s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.614464201 (8h14m14.55288005s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.646281806 (8h14m14.584697655s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.684232226 (8h14m14.622648075s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.736100365 (8h14m14.674516214s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.744475628 (8h14m14.682891477s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.775567917 (8h14m14.713983766s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.785843175 (8h14m14.724259025s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.793127933 (8h14m14.731543773s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.805200522 (8h14m14.743616371s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.840123841 (8h14m14.77853969s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.895304316 (8h14m14.833720165s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915155747 (8h14m14.853571596s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915179071 (8h14m14.85359492s) [seg_pos={0 0}, upload=7849]: Segment reached optimal threshold of 13 pieces. Aug 6 22:23:06.915296927 (8h14m14.853712816s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 6 22:23:06.915493824 (8h14m14.853909673s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection Aug 6 22:23:06.915533600 (8h14m14.853949449s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: read tcp 188.166.225.170:33788->104.36.82.3:4767: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 6 22:23:06.915551634 (8h14m14.853967483s) [seg_pos={0 0}, upload=7849]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:23:06.915610066 (8h14m14.854025915s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:23:06.915613933 (8h14m14.854029772s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:23:07.212500018 (8h14m15.150915887s) [upload=7849]: Done uploading segments... Aug 6 22:23:07.212522912 (8h14m15.150938751s) [upload=7849]: Upload finished. err: <nil> Duration: 6.91335361s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...................X.............X.......X..................] duration: 2.129766009s FAIL (2025-08-06 22:58:28 UTC) orbiter warning: operation: Commit uplink: too many requests Duration: 2.129813351s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.970617012s FAIL (2025-08-06 22:58:05 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/4M") Duration: 6.970643101s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.524124918s FAIL (2025-08-06 22:58:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== VGIBWSTNBG3T5ZAPE5AQ, seg_pos={0 0}, upload=7839]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: context canceled; context canceled Aug 6 22:22:24.227847111 (8h13m32.16626296s) [seg_pos={0 0}, upload=7839]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:22:24.227953294 (8h13m32.166369183s) [upload=7839]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:22:24.227978793 (8h13m32.166394642s) [upload=7839]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:22:24.511796264 (8h13m32.450212103s) [upload=7839]: Done uploading segments... Aug 6 22:22:24.511838736 (8h13m32.450254585s) [upload=7839]: Upload finished. err: <nil> Aug 6 22:23:04.224450786 (8h14m12.162866635s) [upload=7849]: Starting upload Aug 6 22:23:04.224529026 (8h14m12.162944865s) [upload=7849]: Done starting upload Aug 6 22:23:04.224618757 (8h14m12.163034606s) [upload=7849]: Uploading segments... Aug 6 22:23:04.224730752 (8h14m12.163146591s) [upload=7849]: Got next segment. Inline: false Aug 6 22:23:04.224756230 (8h14m12.163172079s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 6 22:23:04.224760679 (8h14m12.163176528s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 6 22:23:04.456921276 (8h14m12.395337125s) [seg_pos={0 0}, upload=7849]: Begin upload segment... Aug 6 22:23:04.456981781 (8h14m12.395397631s) [seg_pos={0 0}, upload=7849]: Done begin upload segment. Aug 6 22:23:04.457004415 (8h14m12.395420264s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457046024 (8h14m12.395461873s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457558906 (8h14m12.395974755s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457926018 (8h14m12.396341867s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458231853 (8h14m12.396647702s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458325132 (8h14m12.396740981s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458575431 (8h14m12.39699128s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458624104 (8h14m12.397039953s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458906444 (8h14m12.397322293s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458920901 (8h14m12.39733676s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459170670 (8h14m12.397586519s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459279838 (8h14m12.397695687s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459426770 (8h14m12.397842619s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459604800 (8h14m12.398020639s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459897921 (8h14m12.39831377s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.460116189 (8h14m12.398532038s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:06.263153507 (8h14m14.201569346s) [upload=7849]: Next returned nil segment Aug 6 22:23:06.263169687 (8h14m14.201585536s) [upload=7849]: Waiting for error group managing segments... Aug 6 22:23:06.593041172 (8h14m14.531457011s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.614464201 (8h14m14.55288005s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.646281806 (8h14m14.584697655s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.684232226 (8h14m14.622648075s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.736100365 (8h14m14.674516214s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.744475628 (8h14m14.682891477s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.775567917 (8h14m14.713983766s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.785843175 (8h14m14.724259025s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.793127933 (8h14m14.731543773s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.805200522 (8h14m14.743616371s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.840123841 (8h14m14.77853969s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.895304316 (8h14m14.833720165s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915155747 (8h14m14.853571596s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915179071 (8h14m14.85359492s) [seg_pos={0 0}, upload=7849]: Segment reached optimal threshold of 13 pieces. Aug 6 22:23:06.915296927 (8h14m14.853712816s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 6 22:23:06.915493824 (8h14m14.853909673s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection Aug 6 22:23:06.915533600 (8h14m14.853949449s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: read tcp 188.166.225.170:33788->104.36.82.3:4767: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 6 22:23:06.915551634 (8h14m14.853967483s) [seg_pos={0 0}, upload=7849]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:23:06.915610066 (8h14m14.854025915s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:23:06.915613933 (8h14m14.854029772s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:23:07.212500018 (8h14m15.150915887s) [upload=7849]: Done uploading segments... Aug 6 22:23:07.212522912 (8h14m15.150938751s) [upload=7849]: Upload finished. err: <nil> Duration: 6.524151549s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [....................X.............X.......X.................] duration: 2.106043203s FAIL (2025-08-06 22:57:28 UTC) orbiter warning: operation: Commit uplink: too many requests Duration: 2.106070876s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.244539016s FAIL (2025-08-06 22:57:21 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/4M") Duration: 6.244560966s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.030681185s FAIL (2025-08-06 22:57:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== VGIBWSTNBG3T5ZAPE5AQ, seg_pos={0 0}, upload=7839]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: context canceled; context canceled Aug 6 22:22:24.227847111 (8h13m32.16626296s) [seg_pos={0 0}, upload=7839]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:22:24.227953294 (8h13m32.166369183s) [upload=7839]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:22:24.227978793 (8h13m32.166394642s) [upload=7839]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:22:24.511796264 (8h13m32.450212103s) [upload=7839]: Done uploading segments... Aug 6 22:22:24.511838736 (8h13m32.450254585s) [upload=7839]: Upload finished. err: <nil> Aug 6 22:23:04.224450786 (8h14m12.162866635s) [upload=7849]: Starting upload Aug 6 22:23:04.224529026 (8h14m12.162944865s) [upload=7849]: Done starting upload Aug 6 22:23:04.224618757 (8h14m12.163034606s) [upload=7849]: Uploading segments... Aug 6 22:23:04.224730752 (8h14m12.163146591s) [upload=7849]: Got next segment. Inline: false Aug 6 22:23:04.224756230 (8h14m12.163172079s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 6 22:23:04.224760679 (8h14m12.163176528s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 6 22:23:04.456921276 (8h14m12.395337125s) [seg_pos={0 0}, upload=7849]: Begin upload segment... Aug 6 22:23:04.456981781 (8h14m12.395397631s) [seg_pos={0 0}, upload=7849]: Done begin upload segment. Aug 6 22:23:04.457004415 (8h14m12.395420264s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457046024 (8h14m12.395461873s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457558906 (8h14m12.395974755s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457926018 (8h14m12.396341867s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458231853 (8h14m12.396647702s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458325132 (8h14m12.396740981s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458575431 (8h14m12.39699128s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458624104 (8h14m12.397039953s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458906444 (8h14m12.397322293s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458920901 (8h14m12.39733676s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459170670 (8h14m12.397586519s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459279838 (8h14m12.397695687s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459426770 (8h14m12.397842619s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459604800 (8h14m12.398020639s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459897921 (8h14m12.39831377s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.460116189 (8h14m12.398532038s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:06.263153507 (8h14m14.201569346s) [upload=7849]: Next returned nil segment Aug 6 22:23:06.263169687 (8h14m14.201585536s) [upload=7849]: Waiting for error group managing segments... Aug 6 22:23:06.593041172 (8h14m14.531457011s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.614464201 (8h14m14.55288005s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.646281806 (8h14m14.584697655s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.684232226 (8h14m14.622648075s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.736100365 (8h14m14.674516214s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.744475628 (8h14m14.682891477s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.775567917 (8h14m14.713983766s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.785843175 (8h14m14.724259025s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.793127933 (8h14m14.731543773s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.805200522 (8h14m14.743616371s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.840123841 (8h14m14.77853969s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.895304316 (8h14m14.833720165s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915155747 (8h14m14.853571596s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915179071 (8h14m14.85359492s) [seg_pos={0 0}, upload=7849]: Segment reached optimal threshold of 13 pieces. Aug 6 22:23:06.915296927 (8h14m14.853712816s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 6 22:23:06.915493824 (8h14m14.853909673s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection Aug 6 22:23:06.915533600 (8h14m14.853949449s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: read tcp 188.166.225.170:33788->104.36.82.3:4767: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 6 22:23:06.915551634 (8h14m14.853967483s) [seg_pos={0 0}, upload=7849]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:23:06.915610066 (8h14m14.854025915s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:23:06.915613933 (8h14m14.854029772s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:23:07.212500018 (8h14m15.150915887s) [upload=7849]: Done uploading segments... Aug 6 22:23:07.212522912 (8h14m15.150938751s) [upload=7849]: Upload finished. err: <nil> Duration: 6.030711674s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.055058897s FAIL (2025-08-06 22:56:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== VGIBWSTNBG3T5ZAPE5AQ, seg_pos={0 0}, upload=7839]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: context canceled; context canceled Aug 6 22:22:24.227847111 (8h13m32.16626296s) [seg_pos={0 0}, upload=7839]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:22:24.227953294 (8h13m32.166369183s) [upload=7839]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:22:24.227978793 (8h13m32.166394642s) [upload=7839]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:22:24.511796264 (8h13m32.450212103s) [upload=7839]: Done uploading segments... Aug 6 22:22:24.511838736 (8h13m32.450254585s) [upload=7839]: Upload finished. err: <nil> Aug 6 22:23:04.224450786 (8h14m12.162866635s) [upload=7849]: Starting upload Aug 6 22:23:04.224529026 (8h14m12.162944865s) [upload=7849]: Done starting upload Aug 6 22:23:04.224618757 (8h14m12.163034606s) [upload=7849]: Uploading segments... Aug 6 22:23:04.224730752 (8h14m12.163146591s) [upload=7849]: Got next segment. Inline: false Aug 6 22:23:04.224756230 (8h14m12.163172079s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 6 22:23:04.224760679 (8h14m12.163176528s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 6 22:23:04.456921276 (8h14m12.395337125s) [seg_pos={0 0}, upload=7849]: Begin upload segment... Aug 6 22:23:04.456981781 (8h14m12.395397631s) [seg_pos={0 0}, upload=7849]: Done begin upload segment. Aug 6 22:23:04.457004415 (8h14m12.395420264s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457046024 (8h14m12.395461873s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457558906 (8h14m12.395974755s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457926018 (8h14m12.396341867s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458231853 (8h14m12.396647702s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458325132 (8h14m12.396740981s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458575431 (8h14m12.39699128s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458624104 (8h14m12.397039953s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458906444 (8h14m12.397322293s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458920901 (8h14m12.39733676s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459170670 (8h14m12.397586519s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459279838 (8h14m12.397695687s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459426770 (8h14m12.397842619s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459604800 (8h14m12.398020639s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459897921 (8h14m12.39831377s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.460116189 (8h14m12.398532038s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:06.263153507 (8h14m14.201569346s) [upload=7849]: Next returned nil segment Aug 6 22:23:06.263169687 (8h14m14.201585536s) [upload=7849]: Waiting for error group managing segments... Aug 6 22:23:06.593041172 (8h14m14.531457011s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.614464201 (8h14m14.55288005s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.646281806 (8h14m14.584697655s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.684232226 (8h14m14.622648075s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.736100365 (8h14m14.674516214s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.744475628 (8h14m14.682891477s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.775567917 (8h14m14.713983766s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.785843175 (8h14m14.724259025s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.793127933 (8h14m14.731543773s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.805200522 (8h14m14.743616371s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.840123841 (8h14m14.77853969s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.895304316 (8h14m14.833720165s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915155747 (8h14m14.853571596s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915179071 (8h14m14.85359492s) [seg_pos={0 0}, upload=7849]: Segment reached optimal threshold of 13 pieces. Aug 6 22:23:06.915296927 (8h14m14.853712816s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 6 22:23:06.915493824 (8h14m14.853909673s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection Aug 6 22:23:06.915533600 (8h14m14.853949449s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: read tcp 188.166.225.170:33788->104.36.82.3:4767: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 6 22:23:06.915551634 (8h14m14.853967483s) [seg_pos={0 0}, upload=7849]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:23:06.915610066 (8h14m14.854025915s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:23:06.915613933 (8h14m14.854029772s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:23:07.212500018 (8h14m15.150915887s) [upload=7849]: Done uploading segments... Aug 6 22:23:07.212522912 (8h14m15.150938751s) [upload=7849]: Upload finished. err: <nil> Duration: 6.05508136s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.....................X.............X.......X................] duration: 1.910610658s FAIL (2025-08-06 22:56:21 UTC) orbiter warning: operation: Commit uplink: too many requests Duration: 1.910635865s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 6.783796573s FAIL (2025-08-06 22:56:09 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/4M") Duration: 6.783824908s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.801532037s FAIL (2025-08-06 22:55:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== VGIBWSTNBG3T5ZAPE5AQ, seg_pos={0 0}, upload=7839]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: context canceled; context canceled Aug 6 22:22:24.227847111 (8h13m32.16626296s) [seg_pos={0 0}, upload=7839]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:22:24.227953294 (8h13m32.166369183s) [upload=7839]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:22:24.227978793 (8h13m32.166394642s) [upload=7839]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:22:24.511796264 (8h13m32.450212103s) [upload=7839]: Done uploading segments... Aug 6 22:22:24.511838736 (8h13m32.450254585s) [upload=7839]: Upload finished. err: <nil> Aug 6 22:23:04.224450786 (8h14m12.162866635s) [upload=7849]: Starting upload Aug 6 22:23:04.224529026 (8h14m12.162944865s) [upload=7849]: Done starting upload Aug 6 22:23:04.224618757 (8h14m12.163034606s) [upload=7849]: Uploading segments... Aug 6 22:23:04.224730752 (8h14m12.163146591s) [upload=7849]: Got next segment. Inline: false Aug 6 22:23:04.224756230 (8h14m12.163172079s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 6 22:23:04.224760679 (8h14m12.163176528s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 6 22:23:04.456921276 (8h14m12.395337125s) [seg_pos={0 0}, upload=7849]: Begin upload segment... Aug 6 22:23:04.456981781 (8h14m12.395397631s) [seg_pos={0 0}, upload=7849]: Done begin upload segment. Aug 6 22:23:04.457004415 (8h14m12.395420264s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457046024 (8h14m12.395461873s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457558906 (8h14m12.395974755s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457926018 (8h14m12.396341867s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458231853 (8h14m12.396647702s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458325132 (8h14m12.396740981s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458575431 (8h14m12.39699128s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458624104 (8h14m12.397039953s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458906444 (8h14m12.397322293s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458920901 (8h14m12.39733676s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459170670 (8h14m12.397586519s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459279838 (8h14m12.397695687s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459426770 (8h14m12.397842619s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459604800 (8h14m12.398020639s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459897921 (8h14m12.39831377s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.460116189 (8h14m12.398532038s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:06.263153507 (8h14m14.201569346s) [upload=7849]: Next returned nil segment Aug 6 22:23:06.263169687 (8h14m14.201585536s) [upload=7849]: Waiting for error group managing segments... Aug 6 22:23:06.593041172 (8h14m14.531457011s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.614464201 (8h14m14.55288005s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.646281806 (8h14m14.584697655s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.684232226 (8h14m14.622648075s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.736100365 (8h14m14.674516214s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.744475628 (8h14m14.682891477s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.775567917 (8h14m14.713983766s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.785843175 (8h14m14.724259025s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.793127933 (8h14m14.731543773s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.805200522 (8h14m14.743616371s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.840123841 (8h14m14.77853969s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.895304316 (8h14m14.833720165s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915155747 (8h14m14.853571596s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915179071 (8h14m14.85359492s) [seg_pos={0 0}, upload=7849]: Segment reached optimal threshold of 13 pieces. Aug 6 22:23:06.915296927 (8h14m14.853712816s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 6 22:23:06.915493824 (8h14m14.853909673s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection Aug 6 22:23:06.915533600 (8h14m14.853949449s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: read tcp 188.166.225.170:33788->104.36.82.3:4767: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 6 22:23:06.915551634 (8h14m14.853967483s) [seg_pos={0 0}, upload=7849]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:23:06.915610066 (8h14m14.854025915s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:23:06.915613933 (8h14m14.854029772s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:23:07.212500018 (8h14m15.150915887s) [upload=7849]: Done uploading segments... Aug 6 22:23:07.212522912 (8h14m15.150938751s) [upload=7849]: Upload finished. err: <nil> Duration: 6.801555812s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [......................X.............X.......X...............] duration: 2.008245034s FAIL (2025-08-06 22:55:43 UTC) orbiter warning: operation: Commit uplink: too many requests Duration: 2.008265012s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 6.547286314s FAIL (2025-08-06 22:55:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== VGIBWSTNBG3T5ZAPE5AQ, seg_pos={0 0}, upload=7839]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: context canceled; context canceled Aug 6 22:22:24.227847111 (8h13m32.16626296s) [seg_pos={0 0}, upload=7839]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:22:24.227953294 (8h13m32.166369183s) [upload=7839]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:22:24.227978793 (8h13m32.166394642s) [upload=7839]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:22:24.511796264 (8h13m32.450212103s) [upload=7839]: Done uploading segments... Aug 6 22:22:24.511838736 (8h13m32.450254585s) [upload=7839]: Upload finished. err: <nil> Aug 6 22:23:04.224450786 (8h14m12.162866635s) [upload=7849]: Starting upload Aug 6 22:23:04.224529026 (8h14m12.162944865s) [upload=7849]: Done starting upload Aug 6 22:23:04.224618757 (8h14m12.163034606s) [upload=7849]: Uploading segments... Aug 6 22:23:04.224730752 (8h14m12.163146591s) [upload=7849]: Got next segment. Inline: false Aug 6 22:23:04.224756230 (8h14m12.163172079s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 6 22:23:04.224760679 (8h14m12.163176528s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 6 22:23:04.456921276 (8h14m12.395337125s) [seg_pos={0 0}, upload=7849]: Begin upload segment... Aug 6 22:23:04.456981781 (8h14m12.395397631s) [seg_pos={0 0}, upload=7849]: Done begin upload segment. Aug 6 22:23:04.457004415 (8h14m12.395420264s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457046024 (8h14m12.395461873s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457558906 (8h14m12.395974755s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457926018 (8h14m12.396341867s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458231853 (8h14m12.396647702s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458325132 (8h14m12.396740981s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458575431 (8h14m12.39699128s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458624104 (8h14m12.397039953s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458906444 (8h14m12.397322293s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458920901 (8h14m12.39733676s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459170670 (8h14m12.397586519s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459279838 (8h14m12.397695687s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459426770 (8h14m12.397842619s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459604800 (8h14m12.398020639s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459897921 (8h14m12.39831377s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.460116189 (8h14m12.398532038s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:06.263153507 (8h14m14.201569346s) [upload=7849]: Next returned nil segment Aug 6 22:23:06.263169687 (8h14m14.201585536s) [upload=7849]: Waiting for error group managing segments... Aug 6 22:23:06.593041172 (8h14m14.531457011s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.614464201 (8h14m14.55288005s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.646281806 (8h14m14.584697655s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.684232226 (8h14m14.622648075s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.736100365 (8h14m14.674516214s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.744475628 (8h14m14.682891477s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.775567917 (8h14m14.713983766s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.785843175 (8h14m14.724259025s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.793127933 (8h14m14.731543773s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.805200522 (8h14m14.743616371s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.840123841 (8h14m14.77853969s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.895304316 (8h14m14.833720165s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915155747 (8h14m14.853571596s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915179071 (8h14m14.85359492s) [seg_pos={0 0}, upload=7849]: Segment reached optimal threshold of 13 pieces. Aug 6 22:23:06.915296927 (8h14m14.853712816s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 6 22:23:06.915493824 (8h14m14.853909673s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection Aug 6 22:23:06.915533600 (8h14m14.853949449s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: read tcp 188.166.225.170:33788->104.36.82.3:4767: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 6 22:23:06.915551634 (8h14m14.853967483s) [seg_pos={0 0}, upload=7849]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:23:06.915610066 (8h14m14.854025915s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:23:06.915613933 (8h14m14.854029772s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:23:07.212500018 (8h14m15.150915887s) [upload=7849]: Done uploading segments... Aug 6 22:23:07.212522912 (8h14m15.150938751s) [upload=7849]: Upload finished. err: <nil> Duration: 6.547304379s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 7.024973741s FAIL (2025-08-06 22:54:58 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/4M") Duration: 7.02499945s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 6.668103791s FAIL (2025-08-06 22:54:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== VGIBWSTNBG3T5ZAPE5AQ, seg_pos={0 0}, upload=7839]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: context canceled; context canceled Aug 6 22:22:24.227847111 (8h13m32.16626296s) [seg_pos={0 0}, upload=7839]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:22:24.227953294 (8h13m32.166369183s) [upload=7839]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:22:24.227978793 (8h13m32.166394642s) [upload=7839]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:22:24.511796264 (8h13m32.450212103s) [upload=7839]: Done uploading segments... Aug 6 22:22:24.511838736 (8h13m32.450254585s) [upload=7839]: Upload finished. err: <nil> Aug 6 22:23:04.224450786 (8h14m12.162866635s) [upload=7849]: Starting upload Aug 6 22:23:04.224529026 (8h14m12.162944865s) [upload=7849]: Done starting upload Aug 6 22:23:04.224618757 (8h14m12.163034606s) [upload=7849]: Uploading segments... Aug 6 22:23:04.224730752 (8h14m12.163146591s) [upload=7849]: Got next segment. Inline: false Aug 6 22:23:04.224756230 (8h14m12.163172079s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 6 22:23:04.224760679 (8h14m12.163176528s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 6 22:23:04.456921276 (8h14m12.395337125s) [seg_pos={0 0}, upload=7849]: Begin upload segment... Aug 6 22:23:04.456981781 (8h14m12.395397631s) [seg_pos={0 0}, upload=7849]: Done begin upload segment. Aug 6 22:23:04.457004415 (8h14m12.395420264s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457046024 (8h14m12.395461873s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457558906 (8h14m12.395974755s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457926018 (8h14m12.396341867s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458231853 (8h14m12.396647702s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458325132 (8h14m12.396740981s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458575431 (8h14m12.39699128s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458624104 (8h14m12.397039953s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458906444 (8h14m12.397322293s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458920901 (8h14m12.39733676s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459170670 (8h14m12.397586519s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459279838 (8h14m12.397695687s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459426770 (8h14m12.397842619s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459604800 (8h14m12.398020639s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459897921 (8h14m12.39831377s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.460116189 (8h14m12.398532038s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:06.263153507 (8h14m14.201569346s) [upload=7849]: Next returned nil segment Aug 6 22:23:06.263169687 (8h14m14.201585536s) [upload=7849]: Waiting for error group managing segments... Aug 6 22:23:06.593041172 (8h14m14.531457011s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.614464201 (8h14m14.55288005s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.646281806 (8h14m14.584697655s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.684232226 (8h14m14.622648075s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.736100365 (8h14m14.674516214s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.744475628 (8h14m14.682891477s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.775567917 (8h14m14.713983766s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.785843175 (8h14m14.724259025s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.793127933 (8h14m14.731543773s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.805200522 (8h14m14.743616371s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.840123841 (8h14m14.77853969s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.895304316 (8h14m14.833720165s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915155747 (8h14m14.853571596s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915179071 (8h14m14.85359492s) [seg_pos={0 0}, upload=7849]: Segment reached optimal threshold of 13 pieces. Aug 6 22:23:06.915296927 (8h14m14.853712816s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 6 22:23:06.915493824 (8h14m14.853909673s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection Aug 6 22:23:06.915533600 (8h14m14.853949449s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: read tcp 188.166.225.170:33788->104.36.82.3:4767: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 6 22:23:06.915551634 (8h14m14.853967483s) [seg_pos={0 0}, upload=7849]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:23:06.915610066 (8h14m14.854025915s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:23:06.915613933 (8h14m14.854029772s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:23:07.212500018 (8h14m15.150915887s) [upload=7849]: Done uploading segments... Aug 6 22:23:07.212522912 (8h14m15.150938751s) [upload=7849]: Upload finished. err: <nil> Duration: 6.668132204s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.......................X.............X.......X..............] duration: 2.057233103s FAIL (2025-08-06 22:54:30 UTC) orbiter warning: operation: Commit uplink: too many requests Duration: 2.057258672s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.73605145s FAIL (2025-08-06 22:54:09 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/4M") Duration: 6.736080726s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 6.454716511s FAIL (2025-08-06 22:53:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== VGIBWSTNBG3T5ZAPE5AQ, seg_pos={0 0}, upload=7839]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: context canceled; context canceled Aug 6 22:22:24.227847111 (8h13m32.16626296s) [seg_pos={0 0}, upload=7839]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:22:24.227953294 (8h13m32.166369183s) [upload=7839]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:22:24.227978793 (8h13m32.166394642s) [upload=7839]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:22:24.511796264 (8h13m32.450212103s) [upload=7839]: Done uploading segments... Aug 6 22:22:24.511838736 (8h13m32.450254585s) [upload=7839]: Upload finished. err: <nil> Aug 6 22:23:04.224450786 (8h14m12.162866635s) [upload=7849]: Starting upload Aug 6 22:23:04.224529026 (8h14m12.162944865s) [upload=7849]: Done starting upload Aug 6 22:23:04.224618757 (8h14m12.163034606s) [upload=7849]: Uploading segments... Aug 6 22:23:04.224730752 (8h14m12.163146591s) [upload=7849]: Got next segment. Inline: false Aug 6 22:23:04.224756230 (8h14m12.163172079s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 6 22:23:04.224760679 (8h14m12.163176528s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 6 22:23:04.456921276 (8h14m12.395337125s) [seg_pos={0 0}, upload=7849]: Begin upload segment... Aug 6 22:23:04.456981781 (8h14m12.395397631s) [seg_pos={0 0}, upload=7849]: Done begin upload segment. Aug 6 22:23:04.457004415 (8h14m12.395420264s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457046024 (8h14m12.395461873s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457558906 (8h14m12.395974755s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457926018 (8h14m12.396341867s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458231853 (8h14m12.396647702s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458325132 (8h14m12.396740981s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458575431 (8h14m12.39699128s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458624104 (8h14m12.397039953s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458906444 (8h14m12.397322293s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458920901 (8h14m12.39733676s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459170670 (8h14m12.397586519s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459279838 (8h14m12.397695687s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459426770 (8h14m12.397842619s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459604800 (8h14m12.398020639s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459897921 (8h14m12.39831377s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.460116189 (8h14m12.398532038s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:06.263153507 (8h14m14.201569346s) [upload=7849]: Next returned nil segment Aug 6 22:23:06.263169687 (8h14m14.201585536s) [upload=7849]: Waiting for error group managing segments... Aug 6 22:23:06.593041172 (8h14m14.531457011s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.614464201 (8h14m14.55288005s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.646281806 (8h14m14.584697655s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.684232226 (8h14m14.622648075s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.736100365 (8h14m14.674516214s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.744475628 (8h14m14.682891477s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.775567917 (8h14m14.713983766s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.785843175 (8h14m14.724259025s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.793127933 (8h14m14.731543773s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.805200522 (8h14m14.743616371s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.840123841 (8h14m14.77853969s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.895304316 (8h14m14.833720165s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915155747 (8h14m14.853571596s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915179071 (8h14m14.85359492s) [seg_pos={0 0}, upload=7849]: Segment reached optimal threshold of 13 pieces. Aug 6 22:23:06.915296927 (8h14m14.853712816s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 6 22:23:06.915493824 (8h14m14.853909673s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection Aug 6 22:23:06.915533600 (8h14m14.853949449s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: read tcp 188.166.225.170:33788->104.36.82.3:4767: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 6 22:23:06.915551634 (8h14m14.853967483s) [seg_pos={0 0}, upload=7849]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:23:06.915610066 (8h14m14.854025915s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:23:06.915613933 (8h14m14.854029772s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:23:07.212500018 (8h14m15.150915887s) [upload=7849]: Done uploading segments... Aug 6 22:23:07.212522912 (8h14m15.150938751s) [upload=7849]: Upload finished. err: <nil> Duration: 6.454797647s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [........................X.............X.......X.............] duration: 2.072020516s FAIL (2025-08-06 22:53:34 UTC) orbiter warning: operation: Commit uplink: too many requests Duration: 2.072047247s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.........................X.............X.......X............] duration: 1.876008209s FAIL (2025-08-06 22:53:13 UTC) orbiter warning: operation: Commit uplink: too many requests Duration: 1.876033627s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 6.728448346s FAIL (2025-08-06 22:53:05 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/4M") Duration: 6.728484044s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 6.353191137s FAIL (2025-08-06 22:52:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== VGIBWSTNBG3T5ZAPE5AQ, seg_pos={0 0}, upload=7839]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: context canceled; context canceled Aug 6 22:22:24.227847111 (8h13m32.16626296s) [seg_pos={0 0}, upload=7839]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:22:24.227953294 (8h13m32.166369183s) [upload=7839]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:22:24.227978793 (8h13m32.166394642s) [upload=7839]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:22:24.511796264 (8h13m32.450212103s) [upload=7839]: Done uploading segments... Aug 6 22:22:24.511838736 (8h13m32.450254585s) [upload=7839]: Upload finished. err: <nil> Aug 6 22:23:04.224450786 (8h14m12.162866635s) [upload=7849]: Starting upload Aug 6 22:23:04.224529026 (8h14m12.162944865s) [upload=7849]: Done starting upload Aug 6 22:23:04.224618757 (8h14m12.163034606s) [upload=7849]: Uploading segments... Aug 6 22:23:04.224730752 (8h14m12.163146591s) [upload=7849]: Got next segment. Inline: false Aug 6 22:23:04.224756230 (8h14m12.163172079s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 6 22:23:04.224760679 (8h14m12.163176528s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 6 22:23:04.456921276 (8h14m12.395337125s) [seg_pos={0 0}, upload=7849]: Begin upload segment... Aug 6 22:23:04.456981781 (8h14m12.395397631s) [seg_pos={0 0}, upload=7849]: Done begin upload segment. Aug 6 22:23:04.457004415 (8h14m12.395420264s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457046024 (8h14m12.395461873s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457558906 (8h14m12.395974755s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457926018 (8h14m12.396341867s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458231853 (8h14m12.396647702s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458325132 (8h14m12.396740981s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458575431 (8h14m12.39699128s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458624104 (8h14m12.397039953s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458906444 (8h14m12.397322293s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458920901 (8h14m12.39733676s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459170670 (8h14m12.397586519s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459279838 (8h14m12.397695687s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459426770 (8h14m12.397842619s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459604800 (8h14m12.398020639s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459897921 (8h14m12.39831377s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.460116189 (8h14m12.398532038s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:06.263153507 (8h14m14.201569346s) [upload=7849]: Next returned nil segment Aug 6 22:23:06.263169687 (8h14m14.201585536s) [upload=7849]: Waiting for error group managing segments... Aug 6 22:23:06.593041172 (8h14m14.531457011s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.614464201 (8h14m14.55288005s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.646281806 (8h14m14.584697655s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.684232226 (8h14m14.622648075s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.736100365 (8h14m14.674516214s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.744475628 (8h14m14.682891477s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.775567917 (8h14m14.713983766s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.785843175 (8h14m14.724259025s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.793127933 (8h14m14.731543773s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.805200522 (8h14m14.743616371s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.840123841 (8h14m14.77853969s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.895304316 (8h14m14.833720165s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915155747 (8h14m14.853571596s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915179071 (8h14m14.85359492s) [seg_pos={0 0}, upload=7849]: Segment reached optimal threshold of 13 pieces. Aug 6 22:23:06.915296927 (8h14m14.853712816s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 6 22:23:06.915493824 (8h14m14.853909673s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection Aug 6 22:23:06.915533600 (8h14m14.853949449s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: read tcp 188.166.225.170:33788->104.36.82.3:4767: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 6 22:23:06.915551634 (8h14m14.853967483s) [seg_pos={0 0}, upload=7849]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:23:06.915610066 (8h14m14.854025915s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:23:06.915613933 (8h14m14.854029772s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:23:07.212500018 (8h14m15.150915887s) [upload=7849]: Done uploading segments... Aug 6 22:23:07.212522912 (8h14m15.150938751s) [upload=7849]: Upload finished. err: <nil> Duration: 6.353216745s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 6.03066731s FAIL (2025-08-06 22:52:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== VGIBWSTNBG3T5ZAPE5AQ, seg_pos={0 0}, upload=7839]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: context canceled; context canceled Aug 6 22:22:24.227847111 (8h13m32.16626296s) [seg_pos={0 0}, upload=7839]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:22:24.227953294 (8h13m32.166369183s) [upload=7839]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:22:24.227978793 (8h13m32.166394642s) [upload=7839]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:22:24.511796264 (8h13m32.450212103s) [upload=7839]: Done uploading segments... Aug 6 22:22:24.511838736 (8h13m32.450254585s) [upload=7839]: Upload finished. err: <nil> Aug 6 22:23:04.224450786 (8h14m12.162866635s) [upload=7849]: Starting upload Aug 6 22:23:04.224529026 (8h14m12.162944865s) [upload=7849]: Done starting upload Aug 6 22:23:04.224618757 (8h14m12.163034606s) [upload=7849]: Uploading segments... Aug 6 22:23:04.224730752 (8h14m12.163146591s) [upload=7849]: Got next segment. Inline: false Aug 6 22:23:04.224756230 (8h14m12.163172079s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 6 22:23:04.224760679 (8h14m12.163176528s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 6 22:23:04.456921276 (8h14m12.395337125s) [seg_pos={0 0}, upload=7849]: Begin upload segment... Aug 6 22:23:04.456981781 (8h14m12.395397631s) [seg_pos={0 0}, upload=7849]: Done begin upload segment. Aug 6 22:23:04.457004415 (8h14m12.395420264s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457046024 (8h14m12.395461873s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457558906 (8h14m12.395974755s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457926018 (8h14m12.396341867s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458231853 (8h14m12.396647702s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458325132 (8h14m12.396740981s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458575431 (8h14m12.39699128s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458624104 (8h14m12.397039953s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458906444 (8h14m12.397322293s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458920901 (8h14m12.39733676s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459170670 (8h14m12.397586519s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459279838 (8h14m12.397695687s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459426770 (8h14m12.397842619s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459604800 (8h14m12.398020639s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459897921 (8h14m12.39831377s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.460116189 (8h14m12.398532038s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:06.263153507 (8h14m14.201569346s) [upload=7849]: Next returned nil segment Aug 6 22:23:06.263169687 (8h14m14.201585536s) [upload=7849]: Waiting for error group managing segments... Aug 6 22:23:06.593041172 (8h14m14.531457011s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.614464201 (8h14m14.55288005s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.646281806 (8h14m14.584697655s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.684232226 (8h14m14.622648075s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.736100365 (8h14m14.674516214s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.744475628 (8h14m14.682891477s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.775567917 (8h14m14.713983766s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.785843175 (8h14m14.724259025s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.793127933 (8h14m14.731543773s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.805200522 (8h14m14.743616371s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.840123841 (8h14m14.77853969s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.895304316 (8h14m14.833720165s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915155747 (8h14m14.853571596s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915179071 (8h14m14.85359492s) [seg_pos={0 0}, upload=7849]: Segment reached optimal threshold of 13 pieces. Aug 6 22:23:06.915296927 (8h14m14.853712816s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 6 22:23:06.915493824 (8h14m14.853909673s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection Aug 6 22:23:06.915533600 (8h14m14.853949449s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: read tcp 188.166.225.170:33788->104.36.82.3:4767: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 6 22:23:06.915551634 (8h14m14.853967483s) [seg_pos={0 0}, upload=7849]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:23:06.915610066 (8h14m14.854025915s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:23:06.915613933 (8h14m14.854029772s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:23:07.212500018 (8h14m15.150915887s) [upload=7849]: Done uploading segments... Aug 6 22:23:07.212522912 (8h14m15.150938751s) [upload=7849]: Upload finished. err: <nil> Duration: 6.030692448s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..........................X.............X.......X...........] duration: 2.109615126s FAIL (2025-08-06 22:52:17 UTC) orbiter warning: operation: Commit uplink: too many requests Duration: 2.109641576s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 6.466208531s FAIL (2025-08-06 22:52:06 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/4M") Duration: 6.466236906s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.491302652s FAIL (2025-08-06 22:51:48 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/4M") Duration: 6.491326307s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 6.136780124s FAIL (2025-08-06 22:51:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== VGIBWSTNBG3T5ZAPE5AQ, seg_pos={0 0}, upload=7839]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: context canceled; context canceled Aug 6 22:22:24.227847111 (8h13m32.16626296s) [seg_pos={0 0}, upload=7839]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:22:24.227953294 (8h13m32.166369183s) [upload=7839]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:22:24.227978793 (8h13m32.166394642s) [upload=7839]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:22:24.511796264 (8h13m32.450212103s) [upload=7839]: Done uploading segments... Aug 6 22:22:24.511838736 (8h13m32.450254585s) [upload=7839]: Upload finished. err: <nil> Aug 6 22:23:04.224450786 (8h14m12.162866635s) [upload=7849]: Starting upload Aug 6 22:23:04.224529026 (8h14m12.162944865s) [upload=7849]: Done starting upload Aug 6 22:23:04.224618757 (8h14m12.163034606s) [upload=7849]: Uploading segments... Aug 6 22:23:04.224730752 (8h14m12.163146591s) [upload=7849]: Got next segment. Inline: false Aug 6 22:23:04.224756230 (8h14m12.163172079s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 6 22:23:04.224760679 (8h14m12.163176528s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 6 22:23:04.456921276 (8h14m12.395337125s) [seg_pos={0 0}, upload=7849]: Begin upload segment... Aug 6 22:23:04.456981781 (8h14m12.395397631s) [seg_pos={0 0}, upload=7849]: Done begin upload segment. Aug 6 22:23:04.457004415 (8h14m12.395420264s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457046024 (8h14m12.395461873s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457558906 (8h14m12.395974755s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457926018 (8h14m12.396341867s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458231853 (8h14m12.396647702s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458325132 (8h14m12.396740981s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458575431 (8h14m12.39699128s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458624104 (8h14m12.397039953s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458906444 (8h14m12.397322293s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458920901 (8h14m12.39733676s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459170670 (8h14m12.397586519s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459279838 (8h14m12.397695687s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459426770 (8h14m12.397842619s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459604800 (8h14m12.398020639s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459897921 (8h14m12.39831377s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.460116189 (8h14m12.398532038s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:06.263153507 (8h14m14.201569346s) [upload=7849]: Next returned nil segment Aug 6 22:23:06.263169687 (8h14m14.201585536s) [upload=7849]: Waiting for error group managing segments... Aug 6 22:23:06.593041172 (8h14m14.531457011s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.614464201 (8h14m14.55288005s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.646281806 (8h14m14.584697655s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.684232226 (8h14m14.622648075s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.736100365 (8h14m14.674516214s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.744475628 (8h14m14.682891477s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.775567917 (8h14m14.713983766s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.785843175 (8h14m14.724259025s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.793127933 (8h14m14.731543773s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.805200522 (8h14m14.743616371s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.840123841 (8h14m14.77853969s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.895304316 (8h14m14.833720165s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915155747 (8h14m14.853571596s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915179071 (8h14m14.85359492s) [seg_pos={0 0}, upload=7849]: Segment reached optimal threshold of 13 pieces. Aug 6 22:23:06.915296927 (8h14m14.853712816s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 6 22:23:06.915493824 (8h14m14.853909673s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection Aug 6 22:23:06.915533600 (8h14m14.853949449s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: read tcp 188.166.225.170:33788->104.36.82.3:4767: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 6 22:23:06.915551634 (8h14m14.853967483s) [seg_pos={0 0}, upload=7849]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:23:06.915610066 (8h14m14.854025915s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:23:06.915613933 (8h14m14.854029772s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:23:07.212500018 (8h14m15.150915887s) [upload=7849]: Done uploading segments... Aug 6 22:23:07.212522912 (8h14m15.150938751s) [upload=7849]: Upload finished. err: <nil> Duration: 6.13680859s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...........................X.............X.......X..........] duration: 2.077660195s FAIL (2025-08-06 22:51:18 UTC) orbiter warning: operation: Commit uplink: too many requests Duration: 2.077684162s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 7.470154285s FAIL (2025-08-06 22:50:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== VGIBWSTNBG3T5ZAPE5AQ, seg_pos={0 0}, upload=7839]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: context canceled; context canceled Aug 6 22:22:24.227847111 (8h13m32.16626296s) [seg_pos={0 0}, upload=7839]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:22:24.227953294 (8h13m32.166369183s) [upload=7839]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:22:24.227978793 (8h13m32.166394642s) [upload=7839]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:22:24.511796264 (8h13m32.450212103s) [upload=7839]: Done uploading segments... Aug 6 22:22:24.511838736 (8h13m32.450254585s) [upload=7839]: Upload finished. err: <nil> Aug 6 22:23:04.224450786 (8h14m12.162866635s) [upload=7849]: Starting upload Aug 6 22:23:04.224529026 (8h14m12.162944865s) [upload=7849]: Done starting upload Aug 6 22:23:04.224618757 (8h14m12.163034606s) [upload=7849]: Uploading segments... Aug 6 22:23:04.224730752 (8h14m12.163146591s) [upload=7849]: Got next segment. Inline: false Aug 6 22:23:04.224756230 (8h14m12.163172079s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 6 22:23:04.224760679 (8h14m12.163176528s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 6 22:23:04.456921276 (8h14m12.395337125s) [seg_pos={0 0}, upload=7849]: Begin upload segment... Aug 6 22:23:04.456981781 (8h14m12.395397631s) [seg_pos={0 0}, upload=7849]: Done begin upload segment. Aug 6 22:23:04.457004415 (8h14m12.395420264s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457046024 (8h14m12.395461873s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457558906 (8h14m12.395974755s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457926018 (8h14m12.396341867s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458231853 (8h14m12.396647702s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458325132 (8h14m12.396740981s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458575431 (8h14m12.39699128s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458624104 (8h14m12.397039953s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458906444 (8h14m12.397322293s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458920901 (8h14m12.39733676s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459170670 (8h14m12.397586519s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459279838 (8h14m12.397695687s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459426770 (8h14m12.397842619s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459604800 (8h14m12.398020639s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459897921 (8h14m12.39831377s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.460116189 (8h14m12.398532038s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:06.263153507 (8h14m14.201569346s) [upload=7849]: Next returned nil segment Aug 6 22:23:06.263169687 (8h14m14.201585536s) [upload=7849]: Waiting for error group managing segments... Aug 6 22:23:06.593041172 (8h14m14.531457011s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.614464201 (8h14m14.55288005s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.646281806 (8h14m14.584697655s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.684232226 (8h14m14.622648075s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.736100365 (8h14m14.674516214s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.744475628 (8h14m14.682891477s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.775567917 (8h14m14.713983766s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.785843175 (8h14m14.724259025s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.793127933 (8h14m14.731543773s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.805200522 (8h14m14.743616371s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.840123841 (8h14m14.77853969s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.895304316 (8h14m14.833720165s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915155747 (8h14m14.853571596s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915179071 (8h14m14.85359492s) [seg_pos={0 0}, upload=7849]: Segment reached optimal threshold of 13 pieces. Aug 6 22:23:06.915296927 (8h14m14.853712816s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 6 22:23:06.915493824 (8h14m14.853909673s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection Aug 6 22:23:06.915533600 (8h14m14.853949449s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: read tcp 188.166.225.170:33788->104.36.82.3:4767: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 6 22:23:06.915551634 (8h14m14.853967483s) [seg_pos={0 0}, upload=7849]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:23:06.915610066 (8h14m14.854025915s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:23:06.915613933 (8h14m14.854029772s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:23:07.212500018 (8h14m15.150915887s) [upload=7849]: Done uploading segments... Aug 6 22:23:07.212522912 (8h14m15.150938751s) [upload=7849]: Upload finished. err: <nil> Duration: 7.470175878s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 6.209572272s FAIL (2025-08-06 22:50:35 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/4M") Duration: 6.209591018s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 6.319426284s FAIL (2025-08-06 22:50:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== VGIBWSTNBG3T5ZAPE5AQ, seg_pos={0 0}, upload=7839]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: context canceled; context canceled Aug 6 22:22:24.227847111 (8h13m32.16626296s) [seg_pos={0 0}, upload=7839]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:22:24.227953294 (8h13m32.166369183s) [upload=7839]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:22:24.227978793 (8h13m32.166394642s) [upload=7839]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:22:24.511796264 (8h13m32.450212103s) [upload=7839]: Done uploading segments... Aug 6 22:22:24.511838736 (8h13m32.450254585s) [upload=7839]: Upload finished. err: <nil> Aug 6 22:23:04.224450786 (8h14m12.162866635s) [upload=7849]: Starting upload Aug 6 22:23:04.224529026 (8h14m12.162944865s) [upload=7849]: Done starting upload Aug 6 22:23:04.224618757 (8h14m12.163034606s) [upload=7849]: Uploading segments... Aug 6 22:23:04.224730752 (8h14m12.163146591s) [upload=7849]: Got next segment. Inline: false Aug 6 22:23:04.224756230 (8h14m12.163172079s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 6 22:23:04.224760679 (8h14m12.163176528s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 6 22:23:04.456921276 (8h14m12.395337125s) [seg_pos={0 0}, upload=7849]: Begin upload segment... Aug 6 22:23:04.456981781 (8h14m12.395397631s) [seg_pos={0 0}, upload=7849]: Done begin upload segment. Aug 6 22:23:04.457004415 (8h14m12.395420264s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457046024 (8h14m12.395461873s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457558906 (8h14m12.395974755s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457926018 (8h14m12.396341867s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458231853 (8h14m12.396647702s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458325132 (8h14m12.396740981s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458575431 (8h14m12.39699128s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458624104 (8h14m12.397039953s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458906444 (8h14m12.397322293s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458920901 (8h14m12.39733676s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459170670 (8h14m12.397586519s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459279838 (8h14m12.397695687s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459426770 (8h14m12.397842619s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459604800 (8h14m12.398020639s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459897921 (8h14m12.39831377s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.460116189 (8h14m12.398532038s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:06.263153507 (8h14m14.201569346s) [upload=7849]: Next returned nil segment Aug 6 22:23:06.263169687 (8h14m14.201585536s) [upload=7849]: Waiting for error group managing segments... Aug 6 22:23:06.593041172 (8h14m14.531457011s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.614464201 (8h14m14.55288005s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.646281806 (8h14m14.584697655s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.684232226 (8h14m14.622648075s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.736100365 (8h14m14.674516214s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.744475628 (8h14m14.682891477s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.775567917 (8h14m14.713983766s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.785843175 (8h14m14.724259025s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.793127933 (8h14m14.731543773s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.805200522 (8h14m14.743616371s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.840123841 (8h14m14.77853969s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.895304316 (8h14m14.833720165s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915155747 (8h14m14.853571596s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915179071 (8h14m14.85359492s) [seg_pos={0 0}, upload=7849]: Segment reached optimal threshold of 13 pieces. Aug 6 22:23:06.915296927 (8h14m14.853712816s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 6 22:23:06.915493824 (8h14m14.853909673s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection Aug 6 22:23:06.915533600 (8h14m14.853949449s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: read tcp 188.166.225.170:33788->104.36.82.3:4767: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 6 22:23:06.915551634 (8h14m14.853967483s) [seg_pos={0 0}, upload=7849]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:23:06.915610066 (8h14m14.854025915s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:23:06.915613933 (8h14m14.854029772s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:23:07.212500018 (8h14m15.150915887s) [upload=7849]: Done uploading segments... Aug 6 22:23:07.212522912 (8h14m15.150938751s) [upload=7849]: Upload finished. err: <nil> Duration: 6.319449147s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [............................X.............X.......X.........] duration: 1.850037664s FAIL (2025-08-06 22:50:03 UTC) orbiter warning: operation: Commit uplink: too many requests Duration: 1.850064505s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 6.323126216s FAIL (2025-08-06 22:49:56 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/4M") Duration: 6.323151143s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 7.179912383s FAIL (2025-08-06 22:49:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== VGIBWSTNBG3T5ZAPE5AQ, seg_pos={0 0}, upload=7839]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: context canceled; context canceled Aug 6 22:22:24.227847111 (8h13m32.16626296s) [seg_pos={0 0}, upload=7839]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:22:24.227953294 (8h13m32.166369183s) [upload=7839]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:22:24.227978793 (8h13m32.166394642s) [upload=7839]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:22:24.511796264 (8h13m32.450212103s) [upload=7839]: Done uploading segments... Aug 6 22:22:24.511838736 (8h13m32.450254585s) [upload=7839]: Upload finished. err: <nil> Aug 6 22:23:04.224450786 (8h14m12.162866635s) [upload=7849]: Starting upload Aug 6 22:23:04.224529026 (8h14m12.162944865s) [upload=7849]: Done starting upload Aug 6 22:23:04.224618757 (8h14m12.163034606s) [upload=7849]: Uploading segments... Aug 6 22:23:04.224730752 (8h14m12.163146591s) [upload=7849]: Got next segment. Inline: false Aug 6 22:23:04.224756230 (8h14m12.163172079s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 6 22:23:04.224760679 (8h14m12.163176528s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 6 22:23:04.456921276 (8h14m12.395337125s) [seg_pos={0 0}, upload=7849]: Begin upload segment... Aug 6 22:23:04.456981781 (8h14m12.395397631s) [seg_pos={0 0}, upload=7849]: Done begin upload segment. Aug 6 22:23:04.457004415 (8h14m12.395420264s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457046024 (8h14m12.395461873s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457558906 (8h14m12.395974755s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457926018 (8h14m12.396341867s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458231853 (8h14m12.396647702s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458325132 (8h14m12.396740981s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458575431 (8h14m12.39699128s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458624104 (8h14m12.397039953s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458906444 (8h14m12.397322293s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458920901 (8h14m12.39733676s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459170670 (8h14m12.397586519s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459279838 (8h14m12.397695687s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459426770 (8h14m12.397842619s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459604800 (8h14m12.398020639s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459897921 (8h14m12.39831377s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.460116189 (8h14m12.398532038s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:06.263153507 (8h14m14.201569346s) [upload=7849]: Next returned nil segment Aug 6 22:23:06.263169687 (8h14m14.201585536s) [upload=7849]: Waiting for error group managing segments... Aug 6 22:23:06.593041172 (8h14m14.531457011s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.614464201 (8h14m14.55288005s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.646281806 (8h14m14.584697655s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.684232226 (8h14m14.622648075s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.736100365 (8h14m14.674516214s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.744475628 (8h14m14.682891477s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.775567917 (8h14m14.713983766s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.785843175 (8h14m14.724259025s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.793127933 (8h14m14.731543773s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.805200522 (8h14m14.743616371s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.840123841 (8h14m14.77853969s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.895304316 (8h14m14.833720165s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915155747 (8h14m14.853571596s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915179071 (8h14m14.85359492s) [seg_pos={0 0}, upload=7849]: Segment reached optimal threshold of 13 pieces. Aug 6 22:23:06.915296927 (8h14m14.853712816s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 6 22:23:06.915493824 (8h14m14.853909673s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection Aug 6 22:23:06.915533600 (8h14m14.853949449s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: read tcp 188.166.225.170:33788->104.36.82.3:4767: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 6 22:23:06.915551634 (8h14m14.853967483s) [seg_pos={0 0}, upload=7849]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:23:06.915610066 (8h14m14.854025915s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:23:06.915613933 (8h14m14.854029772s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:23:07.212500018 (8h14m15.150915887s) [upload=7849]: Done uploading segments... Aug 6 22:23:07.212522912 (8h14m15.150938751s) [upload=7849]: Upload finished. err: <nil> Duration: 7.179939084s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 6.568333873s FAIL (2025-08-06 22:49:03 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/4M") Duration: 6.568359202s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.............................X.............X.......X........] duration: 2.105545558s FAIL (2025-08-06 22:48:53 UTC) orbiter warning: operation: Commit uplink: too many requests Duration: 2.105567479s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 6.318654895s FAIL (2025-08-06 22:48:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== VGIBWSTNBG3T5ZAPE5AQ, seg_pos={0 0}, upload=7839]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: context canceled; context canceled Aug 6 22:22:24.227847111 (8h13m32.16626296s) [seg_pos={0 0}, upload=7839]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:22:24.227953294 (8h13m32.166369183s) [upload=7839]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:22:24.227978793 (8h13m32.166394642s) [upload=7839]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:22:24.511796264 (8h13m32.450212103s) [upload=7839]: Done uploading segments... Aug 6 22:22:24.511838736 (8h13m32.450254585s) [upload=7839]: Upload finished. err: <nil> Aug 6 22:23:04.224450786 (8h14m12.162866635s) [upload=7849]: Starting upload Aug 6 22:23:04.224529026 (8h14m12.162944865s) [upload=7849]: Done starting upload Aug 6 22:23:04.224618757 (8h14m12.163034606s) [upload=7849]: Uploading segments... Aug 6 22:23:04.224730752 (8h14m12.163146591s) [upload=7849]: Got next segment. Inline: false Aug 6 22:23:04.224756230 (8h14m12.163172079s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 6 22:23:04.224760679 (8h14m12.163176528s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 6 22:23:04.456921276 (8h14m12.395337125s) [seg_pos={0 0}, upload=7849]: Begin upload segment... Aug 6 22:23:04.456981781 (8h14m12.395397631s) [seg_pos={0 0}, upload=7849]: Done begin upload segment. Aug 6 22:23:04.457004415 (8h14m12.395420264s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457046024 (8h14m12.395461873s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457558906 (8h14m12.395974755s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457926018 (8h14m12.396341867s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458231853 (8h14m12.396647702s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458325132 (8h14m12.396740981s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458575431 (8h14m12.39699128s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458624104 (8h14m12.397039953s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458906444 (8h14m12.397322293s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458920901 (8h14m12.39733676s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459170670 (8h14m12.397586519s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459279838 (8h14m12.397695687s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459426770 (8h14m12.397842619s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459604800 (8h14m12.398020639s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459897921 (8h14m12.39831377s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.460116189 (8h14m12.398532038s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:06.263153507 (8h14m14.201569346s) [upload=7849]: Next returned nil segment Aug 6 22:23:06.263169687 (8h14m14.201585536s) [upload=7849]: Waiting for error group managing segments... Aug 6 22:23:06.593041172 (8h14m14.531457011s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.614464201 (8h14m14.55288005s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.646281806 (8h14m14.584697655s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.684232226 (8h14m14.622648075s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.736100365 (8h14m14.674516214s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.744475628 (8h14m14.682891477s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.775567917 (8h14m14.713983766s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.785843175 (8h14m14.724259025s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.793127933 (8h14m14.731543773s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.805200522 (8h14m14.743616371s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.840123841 (8h14m14.77853969s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.895304316 (8h14m14.833720165s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915155747 (8h14m14.853571596s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915179071 (8h14m14.85359492s) [seg_pos={0 0}, upload=7849]: Segment reached optimal threshold of 13 pieces. Aug 6 22:23:06.915296927 (8h14m14.853712816s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 6 22:23:06.915493824 (8h14m14.853909673s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection Aug 6 22:23:06.915533600 (8h14m14.853949449s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: read tcp 188.166.225.170:33788->104.36.82.3:4767: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 6 22:23:06.915551634 (8h14m14.853967483s) [seg_pos={0 0}, upload=7849]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:23:06.915610066 (8h14m14.854025915s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:23:06.915613933 (8h14m14.854029772s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:23:07.212500018 (8h14m15.150915887s) [upload=7849]: Done uploading segments... Aug 6 22:23:07.212522912 (8h14m15.150938751s) [upload=7849]: Upload finished. err: <nil> Duration: 6.318680206s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 6.147385144s FAIL (2025-08-06 22:48:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== VGIBWSTNBG3T5ZAPE5AQ, seg_pos={0 0}, upload=7839]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: context canceled; context canceled Aug 6 22:22:24.227847111 (8h13m32.16626296s) [seg_pos={0 0}, upload=7839]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:22:24.227953294 (8h13m32.166369183s) [upload=7839]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:22:24.227978793 (8h13m32.166394642s) [upload=7839]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:22:24.511796264 (8h13m32.450212103s) [upload=7839]: Done uploading segments... Aug 6 22:22:24.511838736 (8h13m32.450254585s) [upload=7839]: Upload finished. err: <nil> Aug 6 22:23:04.224450786 (8h14m12.162866635s) [upload=7849]: Starting upload Aug 6 22:23:04.224529026 (8h14m12.162944865s) [upload=7849]: Done starting upload Aug 6 22:23:04.224618757 (8h14m12.163034606s) [upload=7849]: Uploading segments... Aug 6 22:23:04.224730752 (8h14m12.163146591s) [upload=7849]: Got next segment. Inline: false Aug 6 22:23:04.224756230 (8h14m12.163172079s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 6 22:23:04.224760679 (8h14m12.163176528s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 6 22:23:04.456921276 (8h14m12.395337125s) [seg_pos={0 0}, upload=7849]: Begin upload segment... Aug 6 22:23:04.456981781 (8h14m12.395397631s) [seg_pos={0 0}, upload=7849]: Done begin upload segment. Aug 6 22:23:04.457004415 (8h14m12.395420264s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457046024 (8h14m12.395461873s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457558906 (8h14m12.395974755s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457926018 (8h14m12.396341867s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458231853 (8h14m12.396647702s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458325132 (8h14m12.396740981s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458575431 (8h14m12.39699128s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458624104 (8h14m12.397039953s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458906444 (8h14m12.397322293s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458920901 (8h14m12.39733676s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459170670 (8h14m12.397586519s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459279838 (8h14m12.397695687s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459426770 (8h14m12.397842619s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459604800 (8h14m12.398020639s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459897921 (8h14m12.39831377s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.460116189 (8h14m12.398532038s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:06.263153507 (8h14m14.201569346s) [upload=7849]: Next returned nil segment Aug 6 22:23:06.263169687 (8h14m14.201585536s) [upload=7849]: Waiting for error group managing segments... Aug 6 22:23:06.593041172 (8h14m14.531457011s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.614464201 (8h14m14.55288005s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.646281806 (8h14m14.584697655s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.684232226 (8h14m14.622648075s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.736100365 (8h14m14.674516214s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.744475628 (8h14m14.682891477s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.775567917 (8h14m14.713983766s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.785843175 (8h14m14.724259025s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.793127933 (8h14m14.731543773s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.805200522 (8h14m14.743616371s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.840123841 (8h14m14.77853969s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.895304316 (8h14m14.833720165s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915155747 (8h14m14.853571596s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915179071 (8h14m14.85359492s) [seg_pos={0 0}, upload=7849]: Segment reached optimal threshold of 13 pieces. Aug 6 22:23:06.915296927 (8h14m14.853712816s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 6 22:23:06.915493824 (8h14m14.853909673s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection Aug 6 22:23:06.915533600 (8h14m14.853949449s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: read tcp 188.166.225.170:33788->104.36.82.3:4767: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 6 22:23:06.915551634 (8h14m14.853967483s) [seg_pos={0 0}, upload=7849]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:23:06.915610066 (8h14m14.854025915s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:23:06.915613933 (8h14m14.854029772s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:23:07.212500018 (8h14m15.150915887s) [upload=7849]: Done uploading segments... Aug 6 22:23:07.212522912 (8h14m15.150938751s) [upload=7849]: Upload finished. err: <nil> Duration: 6.14740929s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..............................X.............X.......X.......] duration: 1.987735265s FAIL (2025-08-06 22:47:55 UTC) orbiter warning: operation: Commit uplink: too many requests Duration: 1.98775853s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 6.656046212s FAIL (2025-08-06 22:47:53 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/4M") Duration: 6.656069236s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 6.957765003s FAIL (2025-08-06 22:47:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== VGIBWSTNBG3T5ZAPE5AQ, seg_pos={0 0}, upload=7839]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e): protocol: context canceled; context canceled Aug 6 22:22:24.227847111 (8h13m32.16626296s) [seg_pos={0 0}, upload=7839]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:22:24.227953294 (8h13m32.166369183s) [upload=7839]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:22:24.227978793 (8h13m32.166394642s) [upload=7839]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:22:24.511796264 (8h13m32.450212103s) [upload=7839]: Done uploading segments... Aug 6 22:22:24.511838736 (8h13m32.450254585s) [upload=7839]: Upload finished. err: <nil> Aug 6 22:23:04.224450786 (8h14m12.162866635s) [upload=7849]: Starting upload Aug 6 22:23:04.224529026 (8h14m12.162944865s) [upload=7849]: Done starting upload Aug 6 22:23:04.224618757 (8h14m12.163034606s) [upload=7849]: Uploading segments... Aug 6 22:23:04.224730752 (8h14m12.163146591s) [upload=7849]: Got next segment. Inline: false Aug 6 22:23:04.224756230 (8h14m12.163172079s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 6 22:23:04.224760679 (8h14m12.163176528s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 6 22:23:04.456921276 (8h14m12.395337125s) [seg_pos={0 0}, upload=7849]: Begin upload segment... Aug 6 22:23:04.456981781 (8h14m12.395397631s) [seg_pos={0 0}, upload=7849]: Done begin upload segment. Aug 6 22:23:04.457004415 (8h14m12.395420264s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457046024 (8h14m12.395461873s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457558906 (8h14m12.395974755s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.457926018 (8h14m12.396341867s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458231853 (8h14m12.396647702s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458325132 (8h14m12.396740981s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458575431 (8h14m12.39699128s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458624104 (8h14m12.397039953s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458906444 (8h14m12.397322293s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.458920901 (8h14m12.39733676s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459170670 (8h14m12.397586519s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459279838 (8h14m12.397695687s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459426770 (8h14m12.397842619s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459604800 (8h14m12.398020639s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.459897921 (8h14m12.39831377s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:04.460116189 (8h14m12.398532038s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Uploading piece... Aug 6 22:23:06.263153507 (8h14m14.201569346s) [upload=7849]: Next returned nil segment Aug 6 22:23:06.263169687 (8h14m14.201585536s) [upload=7849]: Waiting for error group managing segments... Aug 6 22:23:06.593041172 (8h14m14.531457011s) [noise=true , address=77.90.31.17:4771 , piece_id=W6RPZPLNICF5OJ57IOCOSFEPA2ROLGJLCGS3WHJ3AIEYKIJNWQNQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.614464201 (8h14m14.55288005s) [noise=true , address=77.90.31.19:4841 , piece_id=ZBSVBU6WFTKXASVXXYWT4GT3MSOP5WMCQBUV6YE5AOYCCEJ5MPIQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.646281806 (8h14m14.584697655s) [noise=true , address=45.91.116.9:4261 , piece_id=ETGKKVWZIQKRHE7VOYK7YYNAVVLM5J4SAZZKHDNXNGCRV7NTMG5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.684232226 (8h14m14.622648075s) [noise=true , address=104.36.82.11:4113 , piece_id=HGUZT3RVVLFQJKABNOMMZR76OWJ5UUQNWNTP5XIZX3E5FXIUSL7A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.736100365 (8h14m14.674516214s) [noise=true , address=104.36.82.5:4838 , piece_id=SUPJYU4EXQW3NGJKFSZVVGD2LFIA44SCWMMUZIO35XYFGQRWWLAQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.744475628 (8h14m14.682891477s) [noise=true , address=77.90.31.21:4878 , piece_id=CJOZY23NGALJWBBKWS57YVSSPCX3LRP255HGWD7MNRK7J2XDTC5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.775567917 (8h14m14.713983766s) [noise=true , address=77.90.31.10:4519 , piece_id=4UO7DK73UJS5BS7NKRIAJQTUNUJSV5Q2I4K53DRC6XSEBCGERNSQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.785843175 (8h14m14.724259025s) [noise=true , address=45.91.116.14:4190 , piece_id=ZE6HGQG2H3JJQYK4BHNY3LTWGV7LMVFLHPIFUAXSZZP723PQRFZA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.793127933 (8h14m14.731543773s) [noise=true , address=45.91.116.12:4227 , piece_id=J42ZCYMEV5FIA2Q2DV2JDHWSFLK4D2B7XYGGW2X7S3T6XKRIORDA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.805200522 (8h14m14.743616371s) [noise=true , address=104.36.82.7:4912 , piece_id=T55YOIIG3W2KVSGZDLVGHHIW2J4ONW7XR6TZ5SDOM2KSZBG6P4QA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.840123841 (8h14m14.77853969s) [noise=true , address=45.91.116.11:4117 , piece_id=S4B7BROXVTGZPQSHUDA3P5OBFRKHFJXVRFTULNKLGBNZ7VLIA2SQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.895304316 (8h14m14.833720165s) [noise=true , address=45.91.116.4:4938 , piece_id=M6TYOYGHNPPOY7WPUDPRGK67IN3V6SNR54DGM4GH5OHE7HFGMUTA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915155747 (8h14m14.853571596s) [noise=true , address=77.90.31.12:4590 , piece_id=ZZZBTCWYIOUN7772Y5NDMLIOBWZRRPS3APLXEO7CNVDG5GBE2E5Q, seg_pos={0 0}, upload=7849]: Done uploading piece. err: <nil> Aug 6 22:23:06.915179071 (8h14m14.85359492s) [seg_pos={0 0}, upload=7849]: Segment reached optimal threshold of 13 pieces. Aug 6 22:23:06.915296927 (8h14m14.853712816s) [noise=true , address=77.90.31.18:4805 , piece_id=6EOOOZDJB2XLT53M3FZ3KNWNVVJANMXRTBNXDATV373CLDEXAN5A, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 6 22:23:06.915493824 (8h14m14.853909673s) [noise=true , address=77.90.31.13:4626 , piece_id=VPJSTULWVUCWYJ252H47BXUSBLAUCXZJUYOOK7X6RXTKTV3HP3TQ, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43736->77.90.31.13:4626: use of closed network connection Aug 6 22:23:06.915533600 (8h14m14.853949449s) [noise=true , address=104.36.82.3:4767 , piece_id=I4S5BOGIOLZAOZL7HWBN3BO6KDQTKDKYLJAQUP3FBL4ICBI4ILSA, seg_pos={0 0}, upload=7849]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: read tcp 188.166.225.170:33788->104.36.82.3:4767: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 6 22:23:06.915551634 (8h14m14.853967483s) [seg_pos={0 0}, upload=7849]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 6 22:23:06.915610066 (8h14m14.854025915s) [upload=7849]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 6 22:23:06.915613933 (8h14m14.854029772s) [upload=7849]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 6 22:23:07.212500018 (8h14m15.150915887s) [upload=7849]: Done uploading segments... Aug 6 22:23:07.212522912 (8h14m15.150938751s) [upload=7849]: Upload finished. err: <nil> Duration: 6.957786274s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...............................X.............X.......X......] duration: 1.999724316s FAIL (2025-08-06 22:47:37 UTC) orbiter warning: operation: Commit uplink: too many requests Duration: 1.999746267s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)