Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 12.69533833s FAIL (2025-06-24 00:33:06 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 12.695363337s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 7.519080697s FAIL (2025-06-24 00:32:45 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 7.519104042s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 13.330579103s FAIL (2025-06-24 00:32:13 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 13.330603479s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 8.272300413s FAIL (2025-06-24 00:31:08 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 8.272326805s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 7.242341675s FAIL (2025-06-24 00:30:00 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 7.2423703s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.849906703s FAIL (2025-06-24 00:29:41 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.849930799s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.747356554s FAIL (2025-06-24 00:29:23 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.747375221s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 7.27106809s FAIL (2025-06-24 00:28:35 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 7.271095173s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 10.794427528s FAIL (2025-06-24 00:28:22 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 10.794456785s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.749875247s FAIL (2025-06-24 00:27:44 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.749914342s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.702856636s FAIL (2025-06-24 00:26:31 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.70290515s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 6.824600204s FAIL (2025-06-24 00:26:03 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.824630092s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.735142012s FAIL (2025-06-24 00:25:23 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.735169987s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 11.484318605s FAIL (2025-06-24 00:24:45 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 11.484342832s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.733738757s FAIL (2025-06-24 00:24:24 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.733761512s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.807115286s FAIL (2025-06-24 00:23:10 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.807137841s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 19.074067782s FAIL (2025-06-24 00:22:45 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 19.074095736s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 14.064233546s FAIL (2025-06-24 00:22:14 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 14.064260639s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 5.778459211s FAIL (2025-06-24 00:20:57 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.778534217s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 5.748010126s FAIL (2025-06-24 00:19:52 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.748036106s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 5.729670902s FAIL (2025-06-24 00:18:49 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.729701932s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 5.731638601s FAIL (2025-06-24 00:17:37 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.731667035s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 7.172807696s FAIL (2025-06-24 00:17:15 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 7.172844357s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 6.680354202s FAIL (2025-06-24 00:16:21 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.68037875s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 19.420808187s FAIL (2025-06-24 00:15:43 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 19.420831313s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.714681171s FAIL (2025-06-24 00:14:25 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.714705458s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 7.268653202s FAIL (2025-06-24 00:13:18 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 7.268680906s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 22.304556766s FAIL (2025-06-24 00:13:08 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 22.304571434s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.232460218s FAIL (2025-06-24 00:12:46 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== item: pb.BatchRequestItem_SegmentCommit Jun 24 00:11:42.157198426 (15h1m16.927839761s) [upload=12048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:11:42.450049193 (15h1m17.220690528s) [upload=12048]: Done uploading segments... Jun 24 00:11:42.450066877 (15h1m17.220708202s) [upload=12048]: Upload finished. err: <nil> Jun 24 00:12:16.443171088 (15h1m51.213812423s) [upload=12057]: Starting upload Jun 24 00:12:16.443229491 (15h1m51.213870826s) [upload=12057]: Done starting upload Jun 24 00:12:16.443319205 (15h1m51.21396054s) [upload=12057]: Uploading segments... Jun 24 00:12:16.443395563 (15h1m51.214036898s) [upload=12057]: Got next segment. Inline: false Jun 24 00:12:16.443450559 (15h1m51.214091894s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 00:12:16.443455449 (15h1m51.214096784s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 00:12:16.675669648 (15h1m51.446310983s) [seg_pos={0 0}, upload=12057]: Begin upload segment... Jun 24 00:12:16.675711580 (15h1m51.446352915s) [seg_pos={0 0}, upload=12057]: Done begin upload segment. Jun 24 00:12:16.675727090 (15h1m51.446368425s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675734424 (15h1m51.446375759s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.675779271 (15h1m51.446420606s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676047310 (15h1m51.446688645s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676362360 (15h1m51.447003695s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676611603 (15h1m51.447252928s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.676852970 (15h1m51.447494295s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677088898 (15h1m51.447730233s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677331437 (15h1m51.447972772s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677512408 (15h1m51.448153733s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677693899 (15h1m51.448335234s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.677889749 (15h1m51.448531084s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678077592 (15h1m51.448718927s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678271658 (15h1m51.448912993s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678456405 (15h1m51.44909774s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:16.678630232 (15h1m51.449271567s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Uploading piece... Jun 24 00:12:19.999055466 (15h1m54.769696801s) [upload=12057]: Next returned nil segment Jun 24 00:12:19.999072028 (15h1m54.769713363s) [upload=12057]: Waiting for error group managing segments... Jun 24 00:12:21.607904160 (15h1m56.378545515s) [noise=true , address=77.90.31.16:4735 , piece_id=ECJCPSCEU4W65OI5BWVYV4NDS4DQQXADVZEIWXLUAMGLCGWQTIUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:22.002262733 (15h1m56.772904068s) [noise=true , address=45.91.116.4:4938 , piece_id=X66EX7YS2BKP6BTUTT33NFYALDGFXRG3TAGFIHBVKUMAQRQIF77A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:23.413239398 (15h1m58.183880743s) [noise=true , address=77.90.31.15:4699 , piece_id=AB26UFQNVXFKPQHU4HXBX3YQJS2AZ5CSOZXRATGSTLNZ5FEDS4OQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:29.782792765 (15h2m4.55343409s) [noise=true , address=104.36.82.12:4150 , piece_id=ZHEGCNIGZE4IDJ5FFPRLDTBSKBJEIY5RGLVPSXKXV2ZTAF2FV56Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.316727859 (15h2m7.087369194s) [noise=true , address=45.91.116.6:4973 , piece_id=5BVW5JUYKZIIHB726X4NNT4ZEE52KGDFK4KADHBNSROYGRSC73GA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.681499920 (15h2m7.452141255s) [noise=true , address=45.91.116.11:4119 , piece_id=U3FQJPY3RWOLTSV7DVG4U6FR3P3UHYTGHQQ4IG7FONH2QOJXUOUA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:32.975780807 (15h2m7.746422143s) [noise=true , address=77.90.31.11:4555 , piece_id=S6WD4BAMIBUMKZSWXTCYBVUHL4TBXMIITQVHZW3UTWXD2EUUE6CQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:33.250070448 (15h2m8.020711773s) [noise=true , address=104.36.82.2:4732 , piece_id=KIGKYCLA2D4WHWOM2QDZ5TNCDCE7YFDPLS2YUWURD7TGKKRZ5A4A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:38.919160377 (15h2m13.689801702s) [noise=true , address=45.91.116.12:5226 , piece_id=M2VQ3W454TDGOZIRHWP3JQXQJLUEAF64G7CVB7BTUFPNPB3N673Q, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:41.005450406 (15h2m15.776091731s) [noise=true , address=45.91.116.9:5261 , piece_id=5GFRKEPC4NXUITR6FTLJQPJ7XNOGHJWRL2HMQGKOMZOVVSXABVQQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:42.842700256 (15h2m17.613341591s) [noise=true , address=45.91.116.10:4083 , piece_id=2PPW63BI72T4VXIWB2367TSJSVHTFRTD5USPEODNHFMGMP3CNYRQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:43.887133340 (15h2m18.657774675s) [noise=true , address=77.90.31.21:4879 , piece_id=ODKHVYV52OD6CTMKE2VMWP6SM33OIF673P4RLSUDR44WKWB3PJ3A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712295207 (15h2m20.482936542s) [noise=true , address=77.90.31.17:4771 , piece_id=PTVN5MFAOI3DIWMYMU5LXM45K4LKM2IXJ2T4E5HY3EPRIREJTM5A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: <nil> Jun 24 00:12:45.712324513 (15h2m20.482965848s) [seg_pos={0 0}, upload=12057]: Segment reached optimal threshold of 13 pieces. Jun 24 00:12:45.712573256 (15h2m20.483214581s) [noise=true , address=45.91.116.14:5190 , piece_id=IKDH4UDTYDKABABAKCYEVUXR3ZTH2U4YOD4HWAIJNKPOC6BPUQSQ, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47338->45.91.116.14:5190: use of closed network connection Jun 24 00:12:45.712594717 (15h2m20.483236052s) [noise=true , address=77.90.31.11:4553 , piece_id=LJMI6TYVG4HS47ZT3F4X2H7D3EWWSNVGDEY7VHFW6MRYTK6EPK6A, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57616->77.90.31.11:4553: use of closed network connection Jun 24 00:12:45.712620728 (15h2m20.483262063s) [noise=true , address=77.90.31.19:4842 , piece_id=NFGO24RAWX6U6H2LBBMOV6YTH5PKBPY7VD44KDCCSK7Q36TNC3KA, seg_pos={0 0}, upload=12057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42702->77.90.31.19:4842: use of closed network connection Jun 24 00:12:45.712648070 (15h2m20.483289405s) [seg_pos={0 0}, upload=12057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 00:12:45.712703899 (15h2m20.483345224s) [upload=12057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 00:12:45.712707816 (15h2m20.483349151s) [upload=12057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 00:12:46.031924157 (15h2m20.802565492s) [upload=12057]: Done uploading segments... Jun 24 00:12:46.031944617 (15h2m20.802585952s) [upload=12057]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.232483945s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.025483438s FAIL (2025-06-23 20:48:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.02549449s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.963422694s FAIL (2025-06-23 20:48:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.963443304s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.936009581s FAIL (2025-06-23 20:46:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.936024719s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.040241885s FAIL (2025-06-23 20:46:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.040260569s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.030680104s FAIL (2025-06-23 20:45:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.030697128s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.962787496s FAIL (2025-06-23 20:45:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.962806172s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.246112435s FAIL (2025-06-23 20:44:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.246134217s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.856336192s FAIL (2025-06-23 20:43:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.856856289s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.902712263s FAIL (2025-06-23 20:43:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.902731539s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.022540811s FAIL (2025-06-23 20:42:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.022559528s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.062243756s FAIL (2025-06-23 20:41:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.062265877s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 5.866821349s FAIL (2025-06-23 20:40:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.866834945s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 6.152867865s FAIL (2025-06-23 20:40:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.153270263s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 6.554611685s FAIL (2025-06-23 20:39:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.554622776s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 6.154574551s FAIL (2025-06-23 20:38:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.154596543s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.567764246s FAIL (2025-06-23 20:38:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.567814312s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 5.978534661s FAIL (2025-06-23 20:37:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.978548599s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.997648773s FAIL (2025-06-23 20:36:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.99767265s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 6.120300429s FAIL (2025-06-23 20:35:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.120327722s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.938712358s FAIL (2025-06-23 20:34:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.938733908s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.954732601s FAIL (2025-06-23 20:33:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.954746367s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)