Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [XX....XX.XXX.....X..X....X......X.XX.X....XX..XXX..X....X.X.] duration: 11.139468895s FAIL (2025-09-06 17:15:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 17:14:48.236033313 (3h46m24.275606483s) [upload=3291]: Starting upload Sep 6 17:14:48.236116913 (3h46m24.275690033s) [upload=3291]: Done starting upload Sep 6 17:14:48.236226152 (3h46m24.275799312s) [upload=3291]: Uploading segments... Sep 6 17:14:48.236562775 (3h46m24.276135895s) [upload=3291]: Got next segment. Inline: false Sep 6 17:14:48.236596179 (3h46m24.276169319s) [upload=3291]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 17:14:48.236602441 (3h46m24.276175561s) [upload=3291]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 17:14:48.502981767 (3h46m24.542554887s) [seg_pos={0 0}, upload=3291]: Begin upload segment... Sep 6 17:14:48.503047192 (3h46m24.542620312s) [seg_pos={0 0}, upload=3291]: Done begin upload segment. Sep 6 17:14:48.503074234 (3h46m24.542647354s) [noise=true , address=45.91.116.5:4299 , piece_id=3SFASXRQNOIGWA7AUJVQ4KEOBCFRNQHFBHO3L5J3BWYT77JNM6PQ, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.503081908 (3h46m24.542655028s) [noise=true , address=45.91.116.8:4048 , piece_id=DFYMMIFDJ2LRB3JEIIBUS3HOFMTDGVDX5BCBS6O2ICGRQGIKQ62Q, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.503129750 (3h46m24.54270287s) [noise=true , address=77.90.31.17:4772 , piece_id=XGRMYY3Z6PSKWO2WXNZ3TN73LMX5JDJ3VB3WIIWOVMBWVPSWSQXA, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.503490550 (3h46m24.54306368s) [noise=true , address=45.91.116.4:4939 , piece_id=NTFJBMK7MP6ES22A2KE6LDBBNICNUTAONJ6CT3L6UIXEVRVNVMYQ, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.503502594 (3h46m24.543075714s) [noise=true , address=45.91.116.9:4261 , piece_id=2MJHQJP2NI52JERNDTDNJLVIJITLOKJRB2I23ZWLXAWV3I6S2AGA, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.503874665 (3h46m24.543447795s) [noise=true , address=77.90.31.10:4518 , piece_id=KWFLST4RPOOZHPBHCHWLE7LR5UZWOUFYZGFCEGVS2M4SP3VW5PGQ, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.504214366 (3h46m24.543787486s) [noise=true , address=45.91.116.6:4974 , piece_id=67LN6GBNZKYD7TWUTPFPOHXE5H4VYZ43TLJ6H32QA7RDQBWZGVGA, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.504574775 (3h46m24.544147905s) [noise=true , address=77.90.31.12:4592 , piece_id=AZFEZPFXK5GKEELZRUVVQCZHAZM34KKYJW23XIIUVJLI76C5M2UQ, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.504935325 (3h46m24.544508445s) [noise=true , address=77.90.31.21:4878 , piece_id=R26D3DYHVC2RZ3KPE7A5FADMRNXDU56V7I7RJQ7O3GJM7FDM4ECQ, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.505174934 (3h46m24.544748054s) [noise=true , address=45.91.116.10:4082 , piece_id=2W5C35DTSQ4QMERHE66KU5D6VBK4UUON4TXR7W6ZDAPPWZKLV3PQ, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.505500787 (3h46m24.545073918s) [noise=true , address=45.91.116.6:4973 , piece_id=5QFALHHJWC4JUQWO4C6FO5SISZ3MUMM6MVKENMWMZ3O63QCJAXVA, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.506004962 (3h46m24.545578082s) [noise=true , address=45.91.116.11:4119 , piece_id=VX43R5PZI5SD5MY3V7PNM4IN6H3ISXNO5AVFJ2R35JUG3AYNIVDA, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.506368828 (3h46m24.545941948s) [noise=true , address=45.91.116.5:4298 , piece_id=NZ3JY6CXEARSMCSG4WOUZ3I7N6CAZ4L5OOOYMMIMZE3X34WFN6ZQ, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.506716223 (3h46m24.546289343s) [noise=true , address=45.91.116.10:4081 , piece_id=IOKEA3E5HZDZEU5EYZ7BYR6COA3YJQBCDZVIQ7VTCZASYS3553IQ, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.507124244 (3h46m24.546697364s) [noise=true , address=77.90.31.14:4661 , piece_id=VQOODWAM5QSOZT5Y6YHBRYT4IJYM7BRDROUWHEWH5ZW3JUH3XBXQ, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.507505413 (3h46m24.547078533s) [noise=true , address=45.91.116.4:4937 , piece_id=J6LQZMPC2D3OO7GIHZSNJVW7OVY5M2RBKAYJD62YA7LNC6XFNXBQ, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:50.274041855 (3h46m26.313614985s) [upload=3291]: Next returned nil segment Sep 6 17:14:50.274077282 (3h46m26.313650402s) [upload=3291]: Waiting for error group managing segments... Sep 6 17:14:50.854885547 (3h46m26.894458698s) [noise=true , address=45.91.116.4:4939 , piece_id=NTFJBMK7MP6ES22A2KE6LDBBNICNUTAONJ6CT3L6UIXEVRVNVMYQ, seg_pos={0 0}, upload=3291]: Done uploading piece. err: <nil> Sep 6 17:14:50.878432565 (3h46m26.918005685s) [noise=true , address=45.91.116.5:4298 , piece_id=NZ3JY6CXEARSMCSG4WOUZ3I7N6CAZ4L5OOOYMMIMZE3X34WFN6ZQ, seg_pos={0 0}, upload=3291]: Done uploading piece. err: <nil> Sep 6 17:14:50.879359398 (3h46m26.918932518s) [noise=true , address=77.90.31.14:4661 , piece_id=VQOODWAM5QSOZT5Y6YHBRYT4IJYM7BRDROUWHEWH5ZW3JUH3XBXQ, seg_pos={0 0}, upload=3291]: Done uploading piece. err: <nil> Sep 6 17:14:50.893256280 (3h46m26.9328294s) [noise=true , address=45.91.116.8:4048 , piece_id=DFYMMIFDJ2LRB3JEIIBUS3HOFMTDGVDX5BCBS6O2ICGRQGIKQ62Q, seg_pos={0 0}, upload=3291]: Done uploading piece. err: <nil> Sep 6 17:14:50.929946325 (3h46m26.969519445s) [noise=true , address=77.90.31.17:4772 , piece_id=XGRMYY3Z6PSKWO2WXNZ3TN73LMX5JDJ3VB3WIIWOVMBWVPSWSQXA, seg_pos={0 0}, upload=3291]: Done uploading piece. err: <nil> Sep 6 17:14:50.963252817 (3h46m27.002825937s) [noise=true , address=45.91.116.11:4119 , piece_id=VX43R5PZI5SD5MY3V7PNM4IN6H3ISXNO5AVFJ2R35JUG3AYNIVDA, seg_pos={0 0}, upload=3291]: Done uploading piece. err: <nil> Sep 6 17:14:50.983402505 (3h46m27.022975625s) [noise=true , address=77.90.31.10:4518 , piece_id=KWFLST4RPOOZHPBHCHWLE7LR5UZWOUFYZGFCEGVS2M4SP3VW5PGQ, seg_pos={0 0}, upload=3291]: Done uploading piece. err: <nil> Sep 6 17:14:50.986362649 (3h46m27.025935769s) [noise=true , address=45.91.116.9:4261 , piece_id=2MJHQJP2NI52JERNDTDNJLVIJITLOKJRB2I23ZWLXAWV3I6S2AGA, seg_pos={0 0}, upload=3291]: Done uploading piece. err: <nil> Sep 6 17:14:51.157307250 (3h46m27.19688036s) [noise=true , address=45.91.116.10:4082 , piece_id=2W5C35DTSQ4QMERHE66KU5D6VBK4UUON4TXR7W6ZDAPPWZKLV3PQ, seg_pos={0 0}, upload=3291]: Done uploading piece. err: <nil> Sep 6 17:14:51.234727443 (3h46m27.274300564s) [noise=true , address=45.91.116.5:4299 , piece_id=3SFASXRQNOIGWA7AUJVQ4KEOBCFRNQHFBHO3L5J3BWYT77JNM6PQ, seg_pos={0 0}, upload=3291]: Done uploading piece. err: <nil> Sep 6 17:14:51.274052221 (3h46m27.313625331s) [noise=true , address=45.91.116.6:4974 , piece_id=67LN6GBNZKYD7TWUTPFPOHXE5H4VYZ43TLJ6H32QA7RDQBWZGVGA, seg_pos={0 0}, upload=3291]: Done uploading piece. err: <nil> Sep 6 17:14:51.315422052 (3h46m27.354995172s) [noise=true , address=45.91.116.10:4081 , piece_id=IOKEA3E5HZDZEU5EYZ7BYR6COA3YJQBCDZVIQ7VTCZASYS3553IQ, seg_pos={0 0}, upload=3291]: Done uploading piece. err: <nil> Sep 6 17:14:51.321217301 (3h46m27.360790421s) [noise=true , address=77.90.31.12:4592 , piece_id=AZFEZPFXK5GKEELZRUVVQCZHAZM34KKYJW23XIIUVJLI76C5M2UQ, seg_pos={0 0}, upload=3291]: Done uploading piece. err: <nil> Sep 6 17:14:51.321240666 (3h46m27.360813776s) [seg_pos={0 0}, upload=3291]: Segment reached optimal threshold of 13 pieces. Sep 6 17:14:51.321348041 (3h46m27.360921151s) [noise=true , address=45.91.116.6:4973 , piece_id=5QFALHHJWC4JUQWO4C6FO5SISZ3MUMM6MVKENMWMZ3O63QCJAXVA, seg_pos={0 0}, upload=3291]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Sep 6 17:14:51.321406002 (3h46m27.360979112s) [noise=true , address=45.91.116.4:4937 , piece_id=J6LQZMPC2D3OO7GIHZSNJVW7OVY5M2RBKAYJD62YA7LNC6XFNXBQ, seg_pos={0 0}, upload=3291]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Sep 6 17:14:51.321503349 (3h46m27.361076469s) [noise=true , address=77.90.31.21:4878 , piece_id=R26D3DYHVC2RZ3KPE7A5FADMRNXDU56V7I7RJQ7O3GJM7FDM4ECQ, seg_pos={0 0}, upload=3291]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: write tcp 188.166.225.170:36586->77.90.31.21:4878: use of closed network connection; context canceled Sep 6 17:14:51.321527175 (3h46m27.361100295s) [seg_pos={0 0}, upload=3291]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 17:14:51.321607108 (3h46m27.361180228s) [upload=3291]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 17:14:51.321611816 (3h46m27.361184936s) [upload=3291]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 17:14:51.646390821 (3h46m27.685963941s) [upload=3291]: Done uploading segments... Sep 6 17:14:51.646408935 (3h46m27.685982055s) [upload=3291]: Upload finished. err: <nil> Duration: 11.13950828s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [.XX....XX.XXX.....X..X....X......X.XX.X....XX..XXX..X....X.X] duration: 30.002722824s FAIL (2025-09-06 17:15:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 17:14:48.236033313 (3h46m24.275606483s) [upload=3291]: Starting upload Sep 6 17:14:48.236116913 (3h46m24.275690033s) [upload=3291]: Done starting upload Sep 6 17:14:48.236226152 (3h46m24.275799312s) [upload=3291]: Uploading segments... Sep 6 17:14:48.236562775 (3h46m24.276135895s) [upload=3291]: Got next segment. Inline: false Sep 6 17:14:48.236596179 (3h46m24.276169319s) [upload=3291]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 17:14:48.236602441 (3h46m24.276175561s) [upload=3291]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 17:14:48.502981767 (3h46m24.542554887s) [seg_pos={0 0}, upload=3291]: Begin upload segment... Sep 6 17:14:48.503047192 (3h46m24.542620312s) [seg_pos={0 0}, upload=3291]: Done begin upload segment. Sep 6 17:14:48.503074234 (3h46m24.542647354s) [noise=true , address=45.91.116.5:4299 , piece_id=3SFASXRQNOIGWA7AUJVQ4KEOBCFRNQHFBHO3L5J3BWYT77JNM6PQ, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.503081908 (3h46m24.542655028s) [noise=true , address=45.91.116.8:4048 , piece_id=DFYMMIFDJ2LRB3JEIIBUS3HOFMTDGVDX5BCBS6O2ICGRQGIKQ62Q, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.503129750 (3h46m24.54270287s) [noise=true , address=77.90.31.17:4772 , piece_id=XGRMYY3Z6PSKWO2WXNZ3TN73LMX5JDJ3VB3WIIWOVMBWVPSWSQXA, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.503490550 (3h46m24.54306368s) [noise=true , address=45.91.116.4:4939 , piece_id=NTFJBMK7MP6ES22A2KE6LDBBNICNUTAONJ6CT3L6UIXEVRVNVMYQ, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.503502594 (3h46m24.543075714s) [noise=true , address=45.91.116.9:4261 , piece_id=2MJHQJP2NI52JERNDTDNJLVIJITLOKJRB2I23ZWLXAWV3I6S2AGA, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.503874665 (3h46m24.543447795s) [noise=true , address=77.90.31.10:4518 , piece_id=KWFLST4RPOOZHPBHCHWLE7LR5UZWOUFYZGFCEGVS2M4SP3VW5PGQ, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.504214366 (3h46m24.543787486s) [noise=true , address=45.91.116.6:4974 , piece_id=67LN6GBNZKYD7TWUTPFPOHXE5H4VYZ43TLJ6H32QA7RDQBWZGVGA, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.504574775 (3h46m24.544147905s) [noise=true , address=77.90.31.12:4592 , piece_id=AZFEZPFXK5GKEELZRUVVQCZHAZM34KKYJW23XIIUVJLI76C5M2UQ, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.504935325 (3h46m24.544508445s) [noise=true , address=77.90.31.21:4878 , piece_id=R26D3DYHVC2RZ3KPE7A5FADMRNXDU56V7I7RJQ7O3GJM7FDM4ECQ, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.505174934 (3h46m24.544748054s) [noise=true , address=45.91.116.10:4082 , piece_id=2W5C35DTSQ4QMERHE66KU5D6VBK4UUON4TXR7W6ZDAPPWZKLV3PQ, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.505500787 (3h46m24.545073918s) [noise=true , address=45.91.116.6:4973 , piece_id=5QFALHHJWC4JUQWO4C6FO5SISZ3MUMM6MVKENMWMZ3O63QCJAXVA, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.506004962 (3h46m24.545578082s) [noise=true , address=45.91.116.11:4119 , piece_id=VX43R5PZI5SD5MY3V7PNM4IN6H3ISXNO5AVFJ2R35JUG3AYNIVDA, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.506368828 (3h46m24.545941948s) [noise=true , address=45.91.116.5:4298 , piece_id=NZ3JY6CXEARSMCSG4WOUZ3I7N6CAZ4L5OOOYMMIMZE3X34WFN6ZQ, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.506716223 (3h46m24.546289343s) [noise=true , address=45.91.116.10:4081 , piece_id=IOKEA3E5HZDZEU5EYZ7BYR6COA3YJQBCDZVIQ7VTCZASYS3553IQ, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.507124244 (3h46m24.546697364s) [noise=true , address=77.90.31.14:4661 , piece_id=VQOODWAM5QSOZT5Y6YHBRYT4IJYM7BRDROUWHEWH5ZW3JUH3XBXQ, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:48.507505413 (3h46m24.547078533s) [noise=true , address=45.91.116.4:4937 , piece_id=J6LQZMPC2D3OO7GIHZSNJVW7OVY5M2RBKAYJD62YA7LNC6XFNXBQ, seg_pos={0 0}, upload=3291]: Uploading piece... Sep 6 17:14:50.274041855 (3h46m26.313614985s) [upload=3291]: Next returned nil segment Sep 6 17:14:50.274077282 (3h46m26.313650402s) [upload=3291]: Waiting for error group managing segments... Sep 6 17:14:50.854885547 (3h46m26.894458698s) [noise=true , address=45.91.116.4:4939 , piece_id=NTFJBMK7MP6ES22A2KE6LDBBNICNUTAONJ6CT3L6UIXEVRVNVMYQ, seg_pos={0 0}, upload=3291]: Done uploading piece. err: <nil> Sep 6 17:14:50.878432565 (3h46m26.918005685s) [noise=true , address=45.91.116.5:4298 , piece_id=NZ3JY6CXEARSMCSG4WOUZ3I7N6CAZ4L5OOOYMMIMZE3X34WFN6ZQ, seg_pos={0 0}, upload=3291]: Done uploading piece. err: <nil> Sep 6 17:14:50.879359398 (3h46m26.918932518s) [noise=true , address=77.90.31.14:4661 , piece_id=VQOODWAM5QSOZT5Y6YHBRYT4IJYM7BRDROUWHEWH5ZW3JUH3XBXQ, seg_pos={0 0}, upload=3291]: Done uploading piece. err: <nil> Sep 6 17:14:50.893256280 (3h46m26.9328294s) [noise=true , address=45.91.116.8:4048 , piece_id=DFYMMIFDJ2LRB3JEIIBUS3HOFMTDGVDX5BCBS6O2ICGRQGIKQ62Q, seg_pos={0 0}, upload=3291]: Done uploading piece. err: <nil> Sep 6 17:14:50.929946325 (3h46m26.969519445s) [noise=true , address=77.90.31.17:4772 , piece_id=XGRMYY3Z6PSKWO2WXNZ3TN73LMX5JDJ3VB3WIIWOVMBWVPSWSQXA, seg_pos={0 0}, upload=3291]: Done uploading piece. err: <nil> Sep 6 17:14:50.963252817 (3h46m27.002825937s) [noise=true , address=45.91.116.11:4119 , piece_id=VX43R5PZI5SD5MY3V7PNM4IN6H3ISXNO5AVFJ2R35JUG3AYNIVDA, seg_pos={0 0}, upload=3291]: Done uploading piece. err: <nil> Sep 6 17:14:50.983402505 (3h46m27.022975625s) [noise=true , address=77.90.31.10:4518 , piece_id=KWFLST4RPOOZHPBHCHWLE7LR5UZWOUFYZGFCEGVS2M4SP3VW5PGQ, seg_pos={0 0}, upload=3291]: Done uploading piece. err: <nil> Sep 6 17:14:50.986362649 (3h46m27.025935769s) [noise=true , address=45.91.116.9:4261 , piece_id=2MJHQJP2NI52JERNDTDNJLVIJITLOKJRB2I23ZWLXAWV3I6S2AGA, seg_pos={0 0}, upload=3291]: Done uploading piece. err: <nil> Sep 6 17:14:51.157307250 (3h46m27.19688036s) [noise=true , address=45.91.116.10:4082 , piece_id=2W5C35DTSQ4QMERHE66KU5D6VBK4UUON4TXR7W6ZDAPPWZKLV3PQ, seg_pos={0 0}, upload=3291]: Done uploading piece. err: <nil> Sep 6 17:14:51.234727443 (3h46m27.274300564s) [noise=true , address=45.91.116.5:4299 , piece_id=3SFASXRQNOIGWA7AUJVQ4KEOBCFRNQHFBHO3L5J3BWYT77JNM6PQ, seg_pos={0 0}, upload=3291]: Done uploading piece. err: <nil> Sep 6 17:14:51.274052221 (3h46m27.313625331s) [noise=true , address=45.91.116.6:4974 , piece_id=67LN6GBNZKYD7TWUTPFPOHXE5H4VYZ43TLJ6H32QA7RDQBWZGVGA, seg_pos={0 0}, upload=3291]: Done uploading piece. err: <nil> Sep 6 17:14:51.315422052 (3h46m27.354995172s) [noise=true , address=45.91.116.10:4081 , piece_id=IOKEA3E5HZDZEU5EYZ7BYR6COA3YJQBCDZVIQ7VTCZASYS3553IQ, seg_pos={0 0}, upload=3291]: Done uploading piece. err: <nil> Sep 6 17:14:51.321217301 (3h46m27.360790421s) [noise=true , address=77.90.31.12:4592 , piece_id=AZFEZPFXK5GKEELZRUVVQCZHAZM34KKYJW23XIIUVJLI76C5M2UQ, seg_pos={0 0}, upload=3291]: Done uploading piece. err: <nil> Sep 6 17:14:51.321240666 (3h46m27.360813776s) [seg_pos={0 0}, upload=3291]: Segment reached optimal threshold of 13 pieces. Sep 6 17:14:51.321348041 (3h46m27.360921151s) [noise=true , address=45.91.116.6:4973 , piece_id=5QFALHHJWC4JUQWO4C6FO5SISZ3MUMM6MVKENMWMZ3O63QCJAXVA, seg_pos={0 0}, upload=3291]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Sep 6 17:14:51.321406002 (3h46m27.360979112s) [noise=true , address=45.91.116.4:4937 , piece_id=J6LQZMPC2D3OO7GIHZSNJVW7OVY5M2RBKAYJD62YA7LNC6XFNXBQ, seg_pos={0 0}, upload=3291]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Sep 6 17:14:51.321503349 (3h46m27.361076469s) [noise=true , address=77.90.31.21:4878 , piece_id=R26D3DYHVC2RZ3KPE7A5FADMRNXDU56V7I7RJQ7O3GJM7FDM4ECQ, seg_pos={0 0}, upload=3291]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: write tcp 188.166.225.170:36586->77.90.31.21:4878: use of closed network connection; context canceled Sep 6 17:14:51.321527175 (3h46m27.361100295s) [seg_pos={0 0}, upload=3291]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 17:14:51.321607108 (3h46m27.361180228s) [upload=3291]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 17:14:51.321611816 (3h46m27.361184936s) [upload=3291]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 17:14:51.646390821 (3h46m27.685963941s) [upload=3291]: Done uploading segments... Sep 6 17:14:51.646408935 (3h46m27.685982055s) [upload=3291]: Upload finished. err: <nil> Duration: 30.00274664s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [X.XX....XX.XXX.....X..X....X......X.XX.X....XX..XXX..X....X.] duration: 10.266284031s FAIL (2025-09-06 17:14:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 17:13:32.828067669 (3h45m8.867640779s) [upload=3271]: Starting upload Sep 6 17:13:32.828122674 (3h45m8.867695794s) [upload=3271]: Done starting upload Sep 6 17:13:32.828210001 (3h45m8.867783121s) [upload=3271]: Uploading segments... Sep 6 17:13:32.828298642 (3h45m8.867871762s) [upload=3271]: Got next segment. Inline: false Sep 6 17:13:32.828350600 (3h45m8.86792372s) [upload=3271]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 17:13:32.828356021 (3h45m8.867929141s) [upload=3271]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 17:13:33.100908278 (3h45m9.140481418s) [seg_pos={0 0}, upload=3271]: Begin upload segment... Sep 6 17:13:33.100975517 (3h45m9.140548637s) [seg_pos={0 0}, upload=3271]: Done begin upload segment. Sep 6 17:13:33.101008190 (3h45m9.14058131s) [noise=true , address=45.91.116.11:4117 , piece_id=JA3TOJC6IDJZRHZRN73M3USGVQTYILIC6XBVOPZFE5BNX77NTTYA, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.101468721 (3h45m9.141041841s) [noise=true , address=45.91.116.11:4119 , piece_id=JU2EHHXL2W57V5VDRT5QWHIKKYDOMVFUCX6OWH5GPRFY6GRTQLIA, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.101561117 (3h45m9.141134237s) [noise=true , address=45.91.116.9:4263 , piece_id=MVMQIF3MJHPUQZ6Q7QNZ6N6GQNUNSB64OKWSLD64G4PYJ73YINRA, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.101880078 (3h45m9.141453198s) [noise=true , address=45.91.116.5:4298 , piece_id=GJ5NQGNYED4GHL24YVXJHI7QN5G4NJ5M2OEDMPC5ENGWOQSSDSXA, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.102055694 (3h45m9.141628814s) [noise=true , address=45.91.116.6:4974 , piece_id=WXQPBWWTKRNIOPCEPY7DRTXWD4ZBWDJBSYC4OHRLQQOCHED3PKOA, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.102149374 (3h45m9.141722484s) [noise=true , address=45.91.116.13:4153 , piece_id=OTNCXTVAPT5WYGJ7YKZ7WF6RJZOR6ACPLV2PTS2S22WGYWNOWYGQ, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.102359856 (3h45m9.141932976s) [noise=true , address=45.91.116.13:4155 , piece_id=542D2YUM5AKYUP2VFG6ST2GSL2NROMF3XLWALUALVH22L342GX5A, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.102372150 (3h45m9.14194527s) [noise=true , address=45.91.116.4:4938 , piece_id=FGUOGCMKEOP4TQQ3FE6PSCJFFBTEPUVEEZM54BBD2DGOBVZXDFAQ, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.102613932 (3h45m9.142187053s) [noise=true , address=45.91.116.14:4190 , piece_id=FWBHZQN4SPLH2I7AYXAKUKYBUFJQUQ4DYQE6IFEUZLLG37VHIYCQ, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.102659790 (3h45m9.14223291s) [noise=true , address=77.90.31.10:4519 , piece_id=6TBKXUTQUJGE6PKX3WG5ZZTZWXJHTG4CKKLCM2ZYT5NGXBVYASRQ, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.102840706 (3h45m9.142413826s) [noise=true , address=77.90.31.18:4806 , piece_id=2MQ2AS4HE6I6A5HRRCCF3CIOYMZNEEUDHKHSCSMNRMZPGDGZ3F3A, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.102880652 (3h45m9.142453772s) [noise=true , address=77.90.31.17:4771 , piece_id=WMO7HEY5HJZX7PI5DNUW3D6O3N3EXOVHJHCVFZVA2OAU3OTFUBIA, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.103067630 (3h45m9.14264075s) [noise=true , address=77.90.31.15:4697 , piece_id=HS4Y6DKUWO7ZDJ4ZUH5CACA5LJQ6GQWO33FNPJXEDYDXGKRSNHHQ, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.103179103 (3h45m9.142752223s) [noise=true , address=77.90.31.16:4735 , piece_id=LZ5UJS2CO5HWEDLHCZICBEOZX5QO7PFGPLSN2T4NGQQTQ3CCL5BQ, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.103291909 (3h45m9.142865029s) [noise=true , address=45.91.116.4:4939 , piece_id=ZB5356QG27ZX6CZV4QRA4NEZMMXXAOFEGW5QSJ53GJIRJCLKPPOQ, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.103569189 (3h45m9.143142309s) [noise=true , address=45.91.116.5:4299 , piece_id=5UEENP4SKWS3HK3EI6NTHIRHQ5UKV557LTABUY5E4RIMTK3PKTIQ, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:34.917192204 (3h45m10.956765324s) [upload=3271]: Next returned nil segment Sep 6 17:13:34.917215738 (3h45m10.956788858s) [upload=3271]: Waiting for error group managing segments... Sep 6 17:13:35.212789218 (3h45m11.252362328s) [noise=true , address=45.91.116.13:4153 , piece_id=OTNCXTVAPT5WYGJ7YKZ7WF6RJZOR6ACPLV2PTS2S22WGYWNOWYGQ, seg_pos={0 0}, upload=3271]: Done uploading piece. err: <nil> Sep 6 17:13:35.215165694 (3h45m11.254738814s) [noise=true , address=45.91.116.9:4263 , piece_id=MVMQIF3MJHPUQZ6Q7QNZ6N6GQNUNSB64OKWSLD64G4PYJ73YINRA, seg_pos={0 0}, upload=3271]: Done uploading piece. err: <nil> Sep 6 17:13:35.230806523 (3h45m11.270379653s) [noise=true , address=45.91.116.4:4938 , piece_id=FGUOGCMKEOP4TQQ3FE6PSCJFFBTEPUVEEZM54BBD2DGOBVZXDFAQ, seg_pos={0 0}, upload=3271]: Done uploading piece. err: <nil> Sep 6 17:13:35.283268218 (3h45m11.322841328s) [noise=true , address=45.91.116.11:4117 , piece_id=JA3TOJC6IDJZRHZRN73M3USGVQTYILIC6XBVOPZFE5BNX77NTTYA, seg_pos={0 0}, upload=3271]: Done uploading piece. err: <nil> Sep 6 17:13:35.289598792 (3h45m11.329171912s) [noise=true , address=45.91.116.11:4119 , piece_id=JU2EHHXL2W57V5VDRT5QWHIKKYDOMVFUCX6OWH5GPRFY6GRTQLIA, seg_pos={0 0}, upload=3271]: Done uploading piece. err: <nil> Sep 6 17:13:35.298759000 (3h45m11.33833212s) [noise=true , address=45.91.116.6:4974 , piece_id=WXQPBWWTKRNIOPCEPY7DRTXWD4ZBWDJBSYC4OHRLQQOCHED3PKOA, seg_pos={0 0}, upload=3271]: Done uploading piece. err: <nil> Sep 6 17:13:35.299166630 (3h45m11.33873975s) [noise=true , address=45.91.116.5:4298 , piece_id=GJ5NQGNYED4GHL24YVXJHI7QN5G4NJ5M2OEDMPC5ENGWOQSSDSXA, seg_pos={0 0}, upload=3271]: Done uploading piece. err: <nil> Sep 6 17:13:35.354579693 (3h45m11.394152813s) [noise=true , address=45.91.116.13:4155 , piece_id=542D2YUM5AKYUP2VFG6ST2GSL2NROMF3XLWALUALVH22L342GX5A, seg_pos={0 0}, upload=3271]: Done uploading piece. err: <nil> Sep 6 17:13:35.371673031 (3h45m11.411246151s) [noise=true , address=77.90.31.15:4697 , piece_id=HS4Y6DKUWO7ZDJ4ZUH5CACA5LJQ6GQWO33FNPJXEDYDXGKRSNHHQ, seg_pos={0 0}, upload=3271]: Done uploading piece. err: <nil> Sep 6 17:13:35.405289626 (3h45m11.444862746s) [noise=true , address=77.90.31.17:4771 , piece_id=WMO7HEY5HJZX7PI5DNUW3D6O3N3EXOVHJHCVFZVA2OAU3OTFUBIA, seg_pos={0 0}, upload=3271]: Done uploading piece. err: <nil> Sep 6 17:13:35.443075459 (3h45m11.482648579s) [noise=true , address=45.91.116.14:4190 , piece_id=FWBHZQN4SPLH2I7AYXAKUKYBUFJQUQ4DYQE6IFEUZLLG37VHIYCQ, seg_pos={0 0}, upload=3271]: Done uploading piece. err: <nil> Sep 6 17:13:35.569397446 (3h45m11.608970566s) [noise=true , address=77.90.31.16:4735 , piece_id=LZ5UJS2CO5HWEDLHCZICBEOZX5QO7PFGPLSN2T4NGQQTQ3CCL5BQ, seg_pos={0 0}, upload=3271]: Done uploading piece. err: <nil> Sep 6 17:13:35.577057684 (3h45m11.616630794s) [noise=true , address=77.90.31.18:4806 , piece_id=2MQ2AS4HE6I6A5HRRCCF3CIOYMZNEEUDHKHSCSMNRMZPGDGZ3F3A, seg_pos={0 0}, upload=3271]: Done uploading piece. err: <nil> Sep 6 17:13:35.577082992 (3h45m11.616656102s) [seg_pos={0 0}, upload=3271]: Segment reached optimal threshold of 13 pieces. Sep 6 17:13:35.577191930 (3h45m11.616765051s) [noise=true , address=45.91.116.5:4299 , piece_id=5UEENP4SKWS3HK3EI6NTHIRHQ5UKV557LTABUY5E4RIMTK3PKTIQ, seg_pos={0 0}, upload=3271]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Sep 6 17:13:35.577422561 (3h45m11.616995681s) [noise=true , address=77.90.31.10:4519 , piece_id=6TBKXUTQUJGE6PKX3WG5ZZTZWXJHTG4CKKLCM2ZYT5NGXBVYASRQ, seg_pos={0 0}, upload=3271]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: rpc: dial tcp 77.90.31.10:4519: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 17:13:35.577510320 (3h45m11.61708344s) [noise=true , address=45.91.116.4:4939 , piece_id=ZB5356QG27ZX6CZV4QRA4NEZMMXXAOFEGW5QSJ53GJIRJCLKPPOQ, seg_pos={0 0}, upload=3271]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 188.166.225.170:41374->45.91.116.4:4939: use of closed network connection; write tcp 188.166.225.170:41374->45.91.116.4:4939: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41374->45.91.116.4:4939: use of closed network connection Sep 6 17:13:35.577570776 (3h45m11.617143916s) [seg_pos={0 0}, upload=3271]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 17:13:35.577681157 (3h45m11.617254287s) [upload=3271]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 17:13:35.577687599 (3h45m11.617260739s) [upload=3271]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 17:13:35.915184067 (3h45m11.954757187s) [upload=3271]: Done uploading segments... Sep 6 17:13:35.915202201 (3h45m11.954775311s) [upload=3271]: Upload finished. err: <nil> Duration: 10.26631394s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [.X.XX....XX.XXX.....X..X....X......X.XX.X....XX..XXX..X....X] duration: 30.002057464s FAIL (2025-09-06 17:14:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 17:13:32.828067669 (3h45m8.867640779s) [upload=3271]: Starting upload Sep 6 17:13:32.828122674 (3h45m8.867695794s) [upload=3271]: Done starting upload Sep 6 17:13:32.828210001 (3h45m8.867783121s) [upload=3271]: Uploading segments... Sep 6 17:13:32.828298642 (3h45m8.867871762s) [upload=3271]: Got next segment. Inline: false Sep 6 17:13:32.828350600 (3h45m8.86792372s) [upload=3271]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 17:13:32.828356021 (3h45m8.867929141s) [upload=3271]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 17:13:33.100908278 (3h45m9.140481418s) [seg_pos={0 0}, upload=3271]: Begin upload segment... Sep 6 17:13:33.100975517 (3h45m9.140548637s) [seg_pos={0 0}, upload=3271]: Done begin upload segment. Sep 6 17:13:33.101008190 (3h45m9.14058131s) [noise=true , address=45.91.116.11:4117 , piece_id=JA3TOJC6IDJZRHZRN73M3USGVQTYILIC6XBVOPZFE5BNX77NTTYA, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.101468721 (3h45m9.141041841s) [noise=true , address=45.91.116.11:4119 , piece_id=JU2EHHXL2W57V5VDRT5QWHIKKYDOMVFUCX6OWH5GPRFY6GRTQLIA, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.101561117 (3h45m9.141134237s) [noise=true , address=45.91.116.9:4263 , piece_id=MVMQIF3MJHPUQZ6Q7QNZ6N6GQNUNSB64OKWSLD64G4PYJ73YINRA, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.101880078 (3h45m9.141453198s) [noise=true , address=45.91.116.5:4298 , piece_id=GJ5NQGNYED4GHL24YVXJHI7QN5G4NJ5M2OEDMPC5ENGWOQSSDSXA, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.102055694 (3h45m9.141628814s) [noise=true , address=45.91.116.6:4974 , piece_id=WXQPBWWTKRNIOPCEPY7DRTXWD4ZBWDJBSYC4OHRLQQOCHED3PKOA, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.102149374 (3h45m9.141722484s) [noise=true , address=45.91.116.13:4153 , piece_id=OTNCXTVAPT5WYGJ7YKZ7WF6RJZOR6ACPLV2PTS2S22WGYWNOWYGQ, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.102359856 (3h45m9.141932976s) [noise=true , address=45.91.116.13:4155 , piece_id=542D2YUM5AKYUP2VFG6ST2GSL2NROMF3XLWALUALVH22L342GX5A, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.102372150 (3h45m9.14194527s) [noise=true , address=45.91.116.4:4938 , piece_id=FGUOGCMKEOP4TQQ3FE6PSCJFFBTEPUVEEZM54BBD2DGOBVZXDFAQ, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.102613932 (3h45m9.142187053s) [noise=true , address=45.91.116.14:4190 , piece_id=FWBHZQN4SPLH2I7AYXAKUKYBUFJQUQ4DYQE6IFEUZLLG37VHIYCQ, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.102659790 (3h45m9.14223291s) [noise=true , address=77.90.31.10:4519 , piece_id=6TBKXUTQUJGE6PKX3WG5ZZTZWXJHTG4CKKLCM2ZYT5NGXBVYASRQ, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.102840706 (3h45m9.142413826s) [noise=true , address=77.90.31.18:4806 , piece_id=2MQ2AS4HE6I6A5HRRCCF3CIOYMZNEEUDHKHSCSMNRMZPGDGZ3F3A, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.102880652 (3h45m9.142453772s) [noise=true , address=77.90.31.17:4771 , piece_id=WMO7HEY5HJZX7PI5DNUW3D6O3N3EXOVHJHCVFZVA2OAU3OTFUBIA, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.103067630 (3h45m9.14264075s) [noise=true , address=77.90.31.15:4697 , piece_id=HS4Y6DKUWO7ZDJ4ZUH5CACA5LJQ6GQWO33FNPJXEDYDXGKRSNHHQ, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.103179103 (3h45m9.142752223s) [noise=true , address=77.90.31.16:4735 , piece_id=LZ5UJS2CO5HWEDLHCZICBEOZX5QO7PFGPLSN2T4NGQQTQ3CCL5BQ, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.103291909 (3h45m9.142865029s) [noise=true , address=45.91.116.4:4939 , piece_id=ZB5356QG27ZX6CZV4QRA4NEZMMXXAOFEGW5QSJ53GJIRJCLKPPOQ, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:33.103569189 (3h45m9.143142309s) [noise=true , address=45.91.116.5:4299 , piece_id=5UEENP4SKWS3HK3EI6NTHIRHQ5UKV557LTABUY5E4RIMTK3PKTIQ, seg_pos={0 0}, upload=3271]: Uploading piece... Sep 6 17:13:34.917192204 (3h45m10.956765324s) [upload=3271]: Next returned nil segment Sep 6 17:13:34.917215738 (3h45m10.956788858s) [upload=3271]: Waiting for error group managing segments... Sep 6 17:13:35.212789218 (3h45m11.252362328s) [noise=true , address=45.91.116.13:4153 , piece_id=OTNCXTVAPT5WYGJ7YKZ7WF6RJZOR6ACPLV2PTS2S22WGYWNOWYGQ, seg_pos={0 0}, upload=3271]: Done uploading piece. err: <nil> Sep 6 17:13:35.215165694 (3h45m11.254738814s) [noise=true , address=45.91.116.9:4263 , piece_id=MVMQIF3MJHPUQZ6Q7QNZ6N6GQNUNSB64OKWSLD64G4PYJ73YINRA, seg_pos={0 0}, upload=3271]: Done uploading piece. err: <nil> Sep 6 17:13:35.230806523 (3h45m11.270379653s) [noise=true , address=45.91.116.4:4938 , piece_id=FGUOGCMKEOP4TQQ3FE6PSCJFFBTEPUVEEZM54BBD2DGOBVZXDFAQ, seg_pos={0 0}, upload=3271]: Done uploading piece. err: <nil> Sep 6 17:13:35.283268218 (3h45m11.322841328s) [noise=true , address=45.91.116.11:4117 , piece_id=JA3TOJC6IDJZRHZRN73M3USGVQTYILIC6XBVOPZFE5BNX77NTTYA, seg_pos={0 0}, upload=3271]: Done uploading piece. err: <nil> Sep 6 17:13:35.289598792 (3h45m11.329171912s) [noise=true , address=45.91.116.11:4119 , piece_id=JU2EHHXL2W57V5VDRT5QWHIKKYDOMVFUCX6OWH5GPRFY6GRTQLIA, seg_pos={0 0}, upload=3271]: Done uploading piece. err: <nil> Sep 6 17:13:35.298759000 (3h45m11.33833212s) [noise=true , address=45.91.116.6:4974 , piece_id=WXQPBWWTKRNIOPCEPY7DRTXWD4ZBWDJBSYC4OHRLQQOCHED3PKOA, seg_pos={0 0}, upload=3271]: Done uploading piece. err: <nil> Sep 6 17:13:35.299166630 (3h45m11.33873975s) [noise=true , address=45.91.116.5:4298 , piece_id=GJ5NQGNYED4GHL24YVXJHI7QN5G4NJ5M2OEDMPC5ENGWOQSSDSXA, seg_pos={0 0}, upload=3271]: Done uploading piece. err: <nil> Sep 6 17:13:35.354579693 (3h45m11.394152813s) [noise=true , address=45.91.116.13:4155 , piece_id=542D2YUM5AKYUP2VFG6ST2GSL2NROMF3XLWALUALVH22L342GX5A, seg_pos={0 0}, upload=3271]: Done uploading piece. err: <nil> Sep 6 17:13:35.371673031 (3h45m11.411246151s) [noise=true , address=77.90.31.15:4697 , piece_id=HS4Y6DKUWO7ZDJ4ZUH5CACA5LJQ6GQWO33FNPJXEDYDXGKRSNHHQ, seg_pos={0 0}, upload=3271]: Done uploading piece. err: <nil> Sep 6 17:13:35.405289626 (3h45m11.444862746s) [noise=true , address=77.90.31.17:4771 , piece_id=WMO7HEY5HJZX7PI5DNUW3D6O3N3EXOVHJHCVFZVA2OAU3OTFUBIA, seg_pos={0 0}, upload=3271]: Done uploading piece. err: <nil> Sep 6 17:13:35.443075459 (3h45m11.482648579s) [noise=true , address=45.91.116.14:4190 , piece_id=FWBHZQN4SPLH2I7AYXAKUKYBUFJQUQ4DYQE6IFEUZLLG37VHIYCQ, seg_pos={0 0}, upload=3271]: Done uploading piece. err: <nil> Sep 6 17:13:35.569397446 (3h45m11.608970566s) [noise=true , address=77.90.31.16:4735 , piece_id=LZ5UJS2CO5HWEDLHCZICBEOZX5QO7PFGPLSN2T4NGQQTQ3CCL5BQ, seg_pos={0 0}, upload=3271]: Done uploading piece. err: <nil> Sep 6 17:13:35.577057684 (3h45m11.616630794s) [noise=true , address=77.90.31.18:4806 , piece_id=2MQ2AS4HE6I6A5HRRCCF3CIOYMZNEEUDHKHSCSMNRMZPGDGZ3F3A, seg_pos={0 0}, upload=3271]: Done uploading piece. err: <nil> Sep 6 17:13:35.577082992 (3h45m11.616656102s) [seg_pos={0 0}, upload=3271]: Segment reached optimal threshold of 13 pieces. Sep 6 17:13:35.577191930 (3h45m11.616765051s) [noise=true , address=45.91.116.5:4299 , piece_id=5UEENP4SKWS3HK3EI6NTHIRHQ5UKV557LTABUY5E4RIMTK3PKTIQ, seg_pos={0 0}, upload=3271]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Sep 6 17:13:35.577422561 (3h45m11.616995681s) [noise=true , address=77.90.31.10:4519 , piece_id=6TBKXUTQUJGE6PKX3WG5ZZTZWXJHTG4CKKLCM2ZYT5NGXBVYASRQ, seg_pos={0 0}, upload=3271]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: rpc: dial tcp 77.90.31.10:4519: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 17:13:35.577510320 (3h45m11.61708344s) [noise=true , address=45.91.116.4:4939 , piece_id=ZB5356QG27ZX6CZV4QRA4NEZMMXXAOFEGW5QSJ53GJIRJCLKPPOQ, seg_pos={0 0}, upload=3271]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 188.166.225.170:41374->45.91.116.4:4939: use of closed network connection; write tcp 188.166.225.170:41374->45.91.116.4:4939: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41374->45.91.116.4:4939: use of closed network connection Sep 6 17:13:35.577570776 (3h45m11.617143916s) [seg_pos={0 0}, upload=3271]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 17:13:35.577681157 (3h45m11.617254287s) [upload=3271]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 17:13:35.577687599 (3h45m11.617260739s) [upload=3271]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 17:13:35.915184067 (3h45m11.954757187s) [upload=3271]: Done uploading segments... Sep 6 17:13:35.915202201 (3h45m11.954775311s) [upload=3271]: Upload finished. err: <nil> Duration: 30.002084727s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [X.X.XX....XX.XXX.....X..X....X......X.XX.X....XX..XXX..X....] duration: 8.873496791s FAIL (2025-09-06 17:12:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 17:10:52.470391152 (3h42m28.509964272s) [upload=3230]: Starting upload Sep 6 17:10:52.470445657 (3h42m28.510018777s) [upload=3230]: Done starting upload Sep 6 17:10:52.470620361 (3h42m28.510193481s) [upload=3230]: Uploading segments... Sep 6 17:10:52.470656090 (3h42m28.51022921s) [upload=3230]: Got next segment. Inline: false Sep 6 17:10:52.470678181 (3h42m28.510251301s) [upload=3230]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 17:10:52.470682490 (3h42m28.5102556s) [upload=3230]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 17:10:52.748377991 (3h42m28.787951152s) [seg_pos={0 0}, upload=3230]: Begin upload segment... Sep 6 17:10:52.748441253 (3h42m28.788014393s) [seg_pos={0 0}, upload=3230]: Done begin upload segment. Sep 6 17:10:52.748513080 (3h42m28.7880862s) [noise=true , address=45.91.116.6:4973 , piece_id=VJLEMOZ5URRVHV2YCUKCIXV3JDHF3FAK32SEDBFLPURLLRSBARHA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.748562254 (3h42m28.788135374s) [noise=true , address=45.91.116.9:4262 , piece_id=5ETIFQYUABHYAD23OABLKMRONKP3FAMOFF4HSRNYNCKYMGQC3MBA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749067632 (3h42m28.788640752s) [noise=true , address=77.90.31.12:4589 , piece_id=RJYFTKEQTI46J5J2VHITLMMOYRH7TSBLBAEH3NZQADQZH7VPJEUA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749075707 (3h42m28.788648817s) [noise=true , address=45.91.116.11:4117 , piece_id=BRRXVJ4ABS762RVT4GEBQPHHDREZRYJDFKNR4WURP55IRBFHVGRA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749355202 (3h42m28.788928322s) [noise=true , address=45.91.116.12:4226 , piece_id=MKGMT47CVESBJPSMQAE7LQ4RNALF5BOQOTVZRFFFFO5Z66EYQLBQ, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749388265 (3h42m28.788961385s) [noise=true , address=45.91.116.4:4938 , piece_id=RYTEHGP6AFZENL7EPTTAGOXJAFIN5S5VQNINZFSK3QK3OLEX6MMA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749663221 (3h42m28.789236341s) [noise=true , address=77.90.31.19:4843 , piece_id=3HMXQELRZRVS5DQ7WW7LIVMCJSQKCDOXKZAWRN5HF2EUZMKTEHXA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749708768 (3h42m28.789281888s) [noise=true , address=45.91.116.7:4010 , piece_id=TGYLFVVW2FQEV3NEVQY5BHLXKM3SJO2GRJYIBIEV5I6CJOERZ34Q, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749967193 (3h42m28.789540313s) [noise=true , address=45.91.116.9:4263 , piece_id=NEZSPOFVK6Z4AOIVRNTINFZ7GDU3IGDTKSTA6O4SBXM7P6QJIJBA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.750066543 (3h42m28.789639663s) [noise=true , address=45.91.116.4:4940 , piece_id=HSXTCVLUG65EQYQNQIU7IFBU4NM2ZBSGTCKBP5PQDOINJZLZ6SRA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.750338233 (3h42m28.789911353s) [noise=true , address=45.91.116.5:4298 , piece_id=JBUC2FAL7DLRRBG5D2MQOSRZ7I4CUA2VFUK6IVFNFNPOT643YSRQ, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.750606877 (3h42m28.790179997s) [noise=true , address=45.91.116.8:4047 , piece_id=3D5N72MAOC62Y7DAEGBF3NXZTCCVKKNLEJM4JH2IQGYKYBFLZGOA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.750851094 (3h42m28.790424214s) [noise=true , address=45.91.116.5:4297 , piece_id=CRXAS52BRF5BWIRTIUCSSJ6H2EGFLMCFRBTCDHGZDMZI4OY5QPAA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.751047630 (3h42m28.79062075s) [noise=true , address=45.91.116.6:4974 , piece_id=QCOOIZ4IYM5EQ6AXV6DZVUYPU3YL75J7S5DWJZKSOA4YVRG6IJEQ, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.752127818 (3h42m28.791700938s) [noise=true , address=45.91.116.7:4009 , piece_id=HTY4BZ3H7UTXHEWYS2BNQKWVOIS4M3LH5ERNX6HVNBJXG7M7XNYQ, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.752356635 (3h42m28.791929755s) [noise=true , address=77.90.31.17:4771 , piece_id=KQJDCQYHVTQLLMURKEMN452LRAO4OHFVSWZYXSB3K5KSGNRNCGPA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:54.590291704 (3h42m30.629864824s) [upload=3230]: Next returned nil segment Sep 6 17:10:54.590312094 (3h42m30.629885204s) [upload=3230]: Waiting for error group managing segments... Sep 6 17:10:54.929472770 (3h42m30.96904589s) [noise=true , address=45.91.116.4:4938 , piece_id=RYTEHGP6AFZENL7EPTTAGOXJAFIN5S5VQNINZFSK3QK3OLEX6MMA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:54.938832018 (3h42m30.978405138s) [noise=true , address=45.91.116.4:4940 , piece_id=HSXTCVLUG65EQYQNQIU7IFBU4NM2ZBSGTCKBP5PQDOINJZLZ6SRA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:54.960049478 (3h42m30.999622598s) [noise=true , address=45.91.116.12:4226 , piece_id=MKGMT47CVESBJPSMQAE7LQ4RNALF5BOQOTVZRFFFFO5Z66EYQLBQ, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:54.966761922 (3h42m31.006335042s) [noise=true , address=45.91.116.9:4262 , piece_id=5ETIFQYUABHYAD23OABLKMRONKP3FAMOFF4HSRNYNCKYMGQC3MBA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:54.970878660 (3h42m31.01045178s) [noise=true , address=45.91.116.6:4974 , piece_id=QCOOIZ4IYM5EQ6AXV6DZVUYPU3YL75J7S5DWJZKSOA4YVRG6IJEQ, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.001147149 (3h42m31.040720269s) [noise=true , address=45.91.116.7:4010 , piece_id=TGYLFVVW2FQEV3NEVQY5BHLXKM3SJO2GRJYIBIEV5I6CJOERZ34Q, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.009058858 (3h42m31.048631978s) [noise=true , address=45.91.116.9:4263 , piece_id=NEZSPOFVK6Z4AOIVRNTINFZ7GDU3IGDTKSTA6O4SBXM7P6QJIJBA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.028713858 (3h42m31.068286978s) [noise=true , address=77.90.31.19:4843 , piece_id=3HMXQELRZRVS5DQ7WW7LIVMCJSQKCDOXKZAWRN5HF2EUZMKTEHXA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.086371005 (3h42m31.125944125s) [noise=true , address=45.91.116.6:4973 , piece_id=VJLEMOZ5URRVHV2YCUKCIXV3JDHF3FAK32SEDBFLPURLLRSBARHA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.097504971 (3h42m31.137078091s) [noise=true , address=45.91.116.8:4047 , piece_id=3D5N72MAOC62Y7DAEGBF3NXZTCCVKKNLEJM4JH2IQGYKYBFLZGOA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.131107198 (3h42m31.170680318s) [noise=true , address=77.90.31.17:4771 , piece_id=KQJDCQYHVTQLLMURKEMN452LRAO4OHFVSWZYXSB3K5KSGNRNCGPA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.173086245 (3h42m31.212659385s) [noise=true , address=45.91.116.5:4298 , piece_id=JBUC2FAL7DLRRBG5D2MQOSRZ7I4CUA2VFUK6IVFNFNPOT643YSRQ, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.188757552 (3h42m31.228330672s) [noise=true , address=45.91.116.7:4009 , piece_id=HTY4BZ3H7UTXHEWYS2BNQKWVOIS4M3LH5ERNX6HVNBJXG7M7XNYQ, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.188782540 (3h42m31.22835566s) [seg_pos={0 0}, upload=3230]: Segment reached optimal threshold of 13 pieces. Sep 6 17:10:55.188896879 (3h42m31.228469999s) [noise=true , address=45.91.116.5:4297 , piece_id=CRXAS52BRF5BWIRTIUCSSJ6H2EGFLMCFRBTCDHGZDMZI4OY5QPAA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Sep 6 17:10:55.188948438 (3h42m31.228521558s) [noise=true , address=45.91.116.11:4117 , piece_id=BRRXVJ4ABS762RVT4GEBQPHHDREZRYJDFKNR4WURP55IRBFHVGRA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 6 17:10:55.189010556 (3h42m31.228583676s) [noise=true , address=77.90.31.12:4589 , piece_id=RJYFTKEQTI46J5J2VHITLMMOYRH7TSBLBAEH3NZQADQZH7VPJEUA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: write tcp 188.166.225.170:53128->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:53128->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53128->77.90.31.12:4589: use of closed network connection Sep 6 17:10:55.189021337 (3h42m31.228594457s) [seg_pos={0 0}, upload=3230]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 17:10:55.189082755 (3h42m31.228655865s) [upload=3230]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 17:10:55.189086422 (3h42m31.228659542s) [upload=3230]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 17:10:55.698217639 (3h42m31.737790759s) [upload=3230]: Done uploading segments... Sep 6 17:10:55.698268134 (3h42m31.737841255s) [upload=3230]: Upload finished. err: <nil> Duration: 8.8735283s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [.X.X.XX....XX.XXX.....X..X....X......X.XX.X....XX..XXX..X...] duration: 10.761019551s FAIL (2025-09-06 17:12:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 17:10:52.470391152 (3h42m28.509964272s) [upload=3230]: Starting upload Sep 6 17:10:52.470445657 (3h42m28.510018777s) [upload=3230]: Done starting upload Sep 6 17:10:52.470620361 (3h42m28.510193481s) [upload=3230]: Uploading segments... Sep 6 17:10:52.470656090 (3h42m28.51022921s) [upload=3230]: Got next segment. Inline: false Sep 6 17:10:52.470678181 (3h42m28.510251301s) [upload=3230]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 17:10:52.470682490 (3h42m28.5102556s) [upload=3230]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 17:10:52.748377991 (3h42m28.787951152s) [seg_pos={0 0}, upload=3230]: Begin upload segment... Sep 6 17:10:52.748441253 (3h42m28.788014393s) [seg_pos={0 0}, upload=3230]: Done begin upload segment. Sep 6 17:10:52.748513080 (3h42m28.7880862s) [noise=true , address=45.91.116.6:4973 , piece_id=VJLEMOZ5URRVHV2YCUKCIXV3JDHF3FAK32SEDBFLPURLLRSBARHA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.748562254 (3h42m28.788135374s) [noise=true , address=45.91.116.9:4262 , piece_id=5ETIFQYUABHYAD23OABLKMRONKP3FAMOFF4HSRNYNCKYMGQC3MBA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749067632 (3h42m28.788640752s) [noise=true , address=77.90.31.12:4589 , piece_id=RJYFTKEQTI46J5J2VHITLMMOYRH7TSBLBAEH3NZQADQZH7VPJEUA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749075707 (3h42m28.788648817s) [noise=true , address=45.91.116.11:4117 , piece_id=BRRXVJ4ABS762RVT4GEBQPHHDREZRYJDFKNR4WURP55IRBFHVGRA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749355202 (3h42m28.788928322s) [noise=true , address=45.91.116.12:4226 , piece_id=MKGMT47CVESBJPSMQAE7LQ4RNALF5BOQOTVZRFFFFO5Z66EYQLBQ, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749388265 (3h42m28.788961385s) [noise=true , address=45.91.116.4:4938 , piece_id=RYTEHGP6AFZENL7EPTTAGOXJAFIN5S5VQNINZFSK3QK3OLEX6MMA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749663221 (3h42m28.789236341s) [noise=true , address=77.90.31.19:4843 , piece_id=3HMXQELRZRVS5DQ7WW7LIVMCJSQKCDOXKZAWRN5HF2EUZMKTEHXA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749708768 (3h42m28.789281888s) [noise=true , address=45.91.116.7:4010 , piece_id=TGYLFVVW2FQEV3NEVQY5BHLXKM3SJO2GRJYIBIEV5I6CJOERZ34Q, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749967193 (3h42m28.789540313s) [noise=true , address=45.91.116.9:4263 , piece_id=NEZSPOFVK6Z4AOIVRNTINFZ7GDU3IGDTKSTA6O4SBXM7P6QJIJBA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.750066543 (3h42m28.789639663s) [noise=true , address=45.91.116.4:4940 , piece_id=HSXTCVLUG65EQYQNQIU7IFBU4NM2ZBSGTCKBP5PQDOINJZLZ6SRA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.750338233 (3h42m28.789911353s) [noise=true , address=45.91.116.5:4298 , piece_id=JBUC2FAL7DLRRBG5D2MQOSRZ7I4CUA2VFUK6IVFNFNPOT643YSRQ, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.750606877 (3h42m28.790179997s) [noise=true , address=45.91.116.8:4047 , piece_id=3D5N72MAOC62Y7DAEGBF3NXZTCCVKKNLEJM4JH2IQGYKYBFLZGOA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.750851094 (3h42m28.790424214s) [noise=true , address=45.91.116.5:4297 , piece_id=CRXAS52BRF5BWIRTIUCSSJ6H2EGFLMCFRBTCDHGZDMZI4OY5QPAA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.751047630 (3h42m28.79062075s) [noise=true , address=45.91.116.6:4974 , piece_id=QCOOIZ4IYM5EQ6AXV6DZVUYPU3YL75J7S5DWJZKSOA4YVRG6IJEQ, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.752127818 (3h42m28.791700938s) [noise=true , address=45.91.116.7:4009 , piece_id=HTY4BZ3H7UTXHEWYS2BNQKWVOIS4M3LH5ERNX6HVNBJXG7M7XNYQ, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.752356635 (3h42m28.791929755s) [noise=true , address=77.90.31.17:4771 , piece_id=KQJDCQYHVTQLLMURKEMN452LRAO4OHFVSWZYXSB3K5KSGNRNCGPA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:54.590291704 (3h42m30.629864824s) [upload=3230]: Next returned nil segment Sep 6 17:10:54.590312094 (3h42m30.629885204s) [upload=3230]: Waiting for error group managing segments... Sep 6 17:10:54.929472770 (3h42m30.96904589s) [noise=true , address=45.91.116.4:4938 , piece_id=RYTEHGP6AFZENL7EPTTAGOXJAFIN5S5VQNINZFSK3QK3OLEX6MMA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:54.938832018 (3h42m30.978405138s) [noise=true , address=45.91.116.4:4940 , piece_id=HSXTCVLUG65EQYQNQIU7IFBU4NM2ZBSGTCKBP5PQDOINJZLZ6SRA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:54.960049478 (3h42m30.999622598s) [noise=true , address=45.91.116.12:4226 , piece_id=MKGMT47CVESBJPSMQAE7LQ4RNALF5BOQOTVZRFFFFO5Z66EYQLBQ, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:54.966761922 (3h42m31.006335042s) [noise=true , address=45.91.116.9:4262 , piece_id=5ETIFQYUABHYAD23OABLKMRONKP3FAMOFF4HSRNYNCKYMGQC3MBA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:54.970878660 (3h42m31.01045178s) [noise=true , address=45.91.116.6:4974 , piece_id=QCOOIZ4IYM5EQ6AXV6DZVUYPU3YL75J7S5DWJZKSOA4YVRG6IJEQ, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.001147149 (3h42m31.040720269s) [noise=true , address=45.91.116.7:4010 , piece_id=TGYLFVVW2FQEV3NEVQY5BHLXKM3SJO2GRJYIBIEV5I6CJOERZ34Q, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.009058858 (3h42m31.048631978s) [noise=true , address=45.91.116.9:4263 , piece_id=NEZSPOFVK6Z4AOIVRNTINFZ7GDU3IGDTKSTA6O4SBXM7P6QJIJBA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.028713858 (3h42m31.068286978s) [noise=true , address=77.90.31.19:4843 , piece_id=3HMXQELRZRVS5DQ7WW7LIVMCJSQKCDOXKZAWRN5HF2EUZMKTEHXA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.086371005 (3h42m31.125944125s) [noise=true , address=45.91.116.6:4973 , piece_id=VJLEMOZ5URRVHV2YCUKCIXV3JDHF3FAK32SEDBFLPURLLRSBARHA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.097504971 (3h42m31.137078091s) [noise=true , address=45.91.116.8:4047 , piece_id=3D5N72MAOC62Y7DAEGBF3NXZTCCVKKNLEJM4JH2IQGYKYBFLZGOA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.131107198 (3h42m31.170680318s) [noise=true , address=77.90.31.17:4771 , piece_id=KQJDCQYHVTQLLMURKEMN452LRAO4OHFVSWZYXSB3K5KSGNRNCGPA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.173086245 (3h42m31.212659385s) [noise=true , address=45.91.116.5:4298 , piece_id=JBUC2FAL7DLRRBG5D2MQOSRZ7I4CUA2VFUK6IVFNFNPOT643YSRQ, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.188757552 (3h42m31.228330672s) [noise=true , address=45.91.116.7:4009 , piece_id=HTY4BZ3H7UTXHEWYS2BNQKWVOIS4M3LH5ERNX6HVNBJXG7M7XNYQ, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.188782540 (3h42m31.22835566s) [seg_pos={0 0}, upload=3230]: Segment reached optimal threshold of 13 pieces. Sep 6 17:10:55.188896879 (3h42m31.228469999s) [noise=true , address=45.91.116.5:4297 , piece_id=CRXAS52BRF5BWIRTIUCSSJ6H2EGFLMCFRBTCDHGZDMZI4OY5QPAA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Sep 6 17:10:55.188948438 (3h42m31.228521558s) [noise=true , address=45.91.116.11:4117 , piece_id=BRRXVJ4ABS762RVT4GEBQPHHDREZRYJDFKNR4WURP55IRBFHVGRA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 6 17:10:55.189010556 (3h42m31.228583676s) [noise=true , address=77.90.31.12:4589 , piece_id=RJYFTKEQTI46J5J2VHITLMMOYRH7TSBLBAEH3NZQADQZH7VPJEUA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: write tcp 188.166.225.170:53128->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:53128->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53128->77.90.31.12:4589: use of closed network connection Sep 6 17:10:55.189021337 (3h42m31.228594457s) [seg_pos={0 0}, upload=3230]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 17:10:55.189082755 (3h42m31.228655865s) [upload=3230]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 17:10:55.189086422 (3h42m31.228659542s) [upload=3230]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 17:10:55.698217639 (3h42m31.737790759s) [upload=3230]: Done uploading segments... Sep 6 17:10:55.698268134 (3h42m31.737841255s) [upload=3230]: Upload finished. err: <nil> Duration: 10.761056443s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [..X.X.XX....XX.XXX.....X..X....X......X.XX.X....XX..XXX..X..] duration: 8.637779855s FAIL (2025-09-06 17:11:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 17:10:52.470391152 (3h42m28.509964272s) [upload=3230]: Starting upload Sep 6 17:10:52.470445657 (3h42m28.510018777s) [upload=3230]: Done starting upload Sep 6 17:10:52.470620361 (3h42m28.510193481s) [upload=3230]: Uploading segments... Sep 6 17:10:52.470656090 (3h42m28.51022921s) [upload=3230]: Got next segment. Inline: false Sep 6 17:10:52.470678181 (3h42m28.510251301s) [upload=3230]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 17:10:52.470682490 (3h42m28.5102556s) [upload=3230]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 17:10:52.748377991 (3h42m28.787951152s) [seg_pos={0 0}, upload=3230]: Begin upload segment... Sep 6 17:10:52.748441253 (3h42m28.788014393s) [seg_pos={0 0}, upload=3230]: Done begin upload segment. Sep 6 17:10:52.748513080 (3h42m28.7880862s) [noise=true , address=45.91.116.6:4973 , piece_id=VJLEMOZ5URRVHV2YCUKCIXV3JDHF3FAK32SEDBFLPURLLRSBARHA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.748562254 (3h42m28.788135374s) [noise=true , address=45.91.116.9:4262 , piece_id=5ETIFQYUABHYAD23OABLKMRONKP3FAMOFF4HSRNYNCKYMGQC3MBA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749067632 (3h42m28.788640752s) [noise=true , address=77.90.31.12:4589 , piece_id=RJYFTKEQTI46J5J2VHITLMMOYRH7TSBLBAEH3NZQADQZH7VPJEUA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749075707 (3h42m28.788648817s) [noise=true , address=45.91.116.11:4117 , piece_id=BRRXVJ4ABS762RVT4GEBQPHHDREZRYJDFKNR4WURP55IRBFHVGRA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749355202 (3h42m28.788928322s) [noise=true , address=45.91.116.12:4226 , piece_id=MKGMT47CVESBJPSMQAE7LQ4RNALF5BOQOTVZRFFFFO5Z66EYQLBQ, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749388265 (3h42m28.788961385s) [noise=true , address=45.91.116.4:4938 , piece_id=RYTEHGP6AFZENL7EPTTAGOXJAFIN5S5VQNINZFSK3QK3OLEX6MMA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749663221 (3h42m28.789236341s) [noise=true , address=77.90.31.19:4843 , piece_id=3HMXQELRZRVS5DQ7WW7LIVMCJSQKCDOXKZAWRN5HF2EUZMKTEHXA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749708768 (3h42m28.789281888s) [noise=true , address=45.91.116.7:4010 , piece_id=TGYLFVVW2FQEV3NEVQY5BHLXKM3SJO2GRJYIBIEV5I6CJOERZ34Q, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749967193 (3h42m28.789540313s) [noise=true , address=45.91.116.9:4263 , piece_id=NEZSPOFVK6Z4AOIVRNTINFZ7GDU3IGDTKSTA6O4SBXM7P6QJIJBA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.750066543 (3h42m28.789639663s) [noise=true , address=45.91.116.4:4940 , piece_id=HSXTCVLUG65EQYQNQIU7IFBU4NM2ZBSGTCKBP5PQDOINJZLZ6SRA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.750338233 (3h42m28.789911353s) [noise=true , address=45.91.116.5:4298 , piece_id=JBUC2FAL7DLRRBG5D2MQOSRZ7I4CUA2VFUK6IVFNFNPOT643YSRQ, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.750606877 (3h42m28.790179997s) [noise=true , address=45.91.116.8:4047 , piece_id=3D5N72MAOC62Y7DAEGBF3NXZTCCVKKNLEJM4JH2IQGYKYBFLZGOA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.750851094 (3h42m28.790424214s) [noise=true , address=45.91.116.5:4297 , piece_id=CRXAS52BRF5BWIRTIUCSSJ6H2EGFLMCFRBTCDHGZDMZI4OY5QPAA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.751047630 (3h42m28.79062075s) [noise=true , address=45.91.116.6:4974 , piece_id=QCOOIZ4IYM5EQ6AXV6DZVUYPU3YL75J7S5DWJZKSOA4YVRG6IJEQ, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.752127818 (3h42m28.791700938s) [noise=true , address=45.91.116.7:4009 , piece_id=HTY4BZ3H7UTXHEWYS2BNQKWVOIS4M3LH5ERNX6HVNBJXG7M7XNYQ, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.752356635 (3h42m28.791929755s) [noise=true , address=77.90.31.17:4771 , piece_id=KQJDCQYHVTQLLMURKEMN452LRAO4OHFVSWZYXSB3K5KSGNRNCGPA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:54.590291704 (3h42m30.629864824s) [upload=3230]: Next returned nil segment Sep 6 17:10:54.590312094 (3h42m30.629885204s) [upload=3230]: Waiting for error group managing segments... Sep 6 17:10:54.929472770 (3h42m30.96904589s) [noise=true , address=45.91.116.4:4938 , piece_id=RYTEHGP6AFZENL7EPTTAGOXJAFIN5S5VQNINZFSK3QK3OLEX6MMA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:54.938832018 (3h42m30.978405138s) [noise=true , address=45.91.116.4:4940 , piece_id=HSXTCVLUG65EQYQNQIU7IFBU4NM2ZBSGTCKBP5PQDOINJZLZ6SRA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:54.960049478 (3h42m30.999622598s) [noise=true , address=45.91.116.12:4226 , piece_id=MKGMT47CVESBJPSMQAE7LQ4RNALF5BOQOTVZRFFFFO5Z66EYQLBQ, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:54.966761922 (3h42m31.006335042s) [noise=true , address=45.91.116.9:4262 , piece_id=5ETIFQYUABHYAD23OABLKMRONKP3FAMOFF4HSRNYNCKYMGQC3MBA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:54.970878660 (3h42m31.01045178s) [noise=true , address=45.91.116.6:4974 , piece_id=QCOOIZ4IYM5EQ6AXV6DZVUYPU3YL75J7S5DWJZKSOA4YVRG6IJEQ, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.001147149 (3h42m31.040720269s) [noise=true , address=45.91.116.7:4010 , piece_id=TGYLFVVW2FQEV3NEVQY5BHLXKM3SJO2GRJYIBIEV5I6CJOERZ34Q, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.009058858 (3h42m31.048631978s) [noise=true , address=45.91.116.9:4263 , piece_id=NEZSPOFVK6Z4AOIVRNTINFZ7GDU3IGDTKSTA6O4SBXM7P6QJIJBA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.028713858 (3h42m31.068286978s) [noise=true , address=77.90.31.19:4843 , piece_id=3HMXQELRZRVS5DQ7WW7LIVMCJSQKCDOXKZAWRN5HF2EUZMKTEHXA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.086371005 (3h42m31.125944125s) [noise=true , address=45.91.116.6:4973 , piece_id=VJLEMOZ5URRVHV2YCUKCIXV3JDHF3FAK32SEDBFLPURLLRSBARHA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.097504971 (3h42m31.137078091s) [noise=true , address=45.91.116.8:4047 , piece_id=3D5N72MAOC62Y7DAEGBF3NXZTCCVKKNLEJM4JH2IQGYKYBFLZGOA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.131107198 (3h42m31.170680318s) [noise=true , address=77.90.31.17:4771 , piece_id=KQJDCQYHVTQLLMURKEMN452LRAO4OHFVSWZYXSB3K5KSGNRNCGPA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.173086245 (3h42m31.212659385s) [noise=true , address=45.91.116.5:4298 , piece_id=JBUC2FAL7DLRRBG5D2MQOSRZ7I4CUA2VFUK6IVFNFNPOT643YSRQ, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.188757552 (3h42m31.228330672s) [noise=true , address=45.91.116.7:4009 , piece_id=HTY4BZ3H7UTXHEWYS2BNQKWVOIS4M3LH5ERNX6HVNBJXG7M7XNYQ, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.188782540 (3h42m31.22835566s) [seg_pos={0 0}, upload=3230]: Segment reached optimal threshold of 13 pieces. Sep 6 17:10:55.188896879 (3h42m31.228469999s) [noise=true , address=45.91.116.5:4297 , piece_id=CRXAS52BRF5BWIRTIUCSSJ6H2EGFLMCFRBTCDHGZDMZI4OY5QPAA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Sep 6 17:10:55.188948438 (3h42m31.228521558s) [noise=true , address=45.91.116.11:4117 , piece_id=BRRXVJ4ABS762RVT4GEBQPHHDREZRYJDFKNR4WURP55IRBFHVGRA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 6 17:10:55.189010556 (3h42m31.228583676s) [noise=true , address=77.90.31.12:4589 , piece_id=RJYFTKEQTI46J5J2VHITLMMOYRH7TSBLBAEH3NZQADQZH7VPJEUA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: write tcp 188.166.225.170:53128->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:53128->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53128->77.90.31.12:4589: use of closed network connection Sep 6 17:10:55.189021337 (3h42m31.228594457s) [seg_pos={0 0}, upload=3230]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 17:10:55.189082755 (3h42m31.228655865s) [upload=3230]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 17:10:55.189086422 (3h42m31.228659542s) [upload=3230]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 17:10:55.698217639 (3h42m31.737790759s) [upload=3230]: Done uploading segments... Sep 6 17:10:55.698268134 (3h42m31.737841255s) [upload=3230]: Upload finished. err: <nil> Duration: 8.637808819s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [X..X.X.XX....XX.XXX.....X..X....X......X.XX.X....XX..XXX..X.] duration: 6.900177133s FAIL (2025-09-06 17:11:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 17:10:52.470391152 (3h42m28.509964272s) [upload=3230]: Starting upload Sep 6 17:10:52.470445657 (3h42m28.510018777s) [upload=3230]: Done starting upload Sep 6 17:10:52.470620361 (3h42m28.510193481s) [upload=3230]: Uploading segments... Sep 6 17:10:52.470656090 (3h42m28.51022921s) [upload=3230]: Got next segment. Inline: false Sep 6 17:10:52.470678181 (3h42m28.510251301s) [upload=3230]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 17:10:52.470682490 (3h42m28.5102556s) [upload=3230]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 17:10:52.748377991 (3h42m28.787951152s) [seg_pos={0 0}, upload=3230]: Begin upload segment... Sep 6 17:10:52.748441253 (3h42m28.788014393s) [seg_pos={0 0}, upload=3230]: Done begin upload segment. Sep 6 17:10:52.748513080 (3h42m28.7880862s) [noise=true , address=45.91.116.6:4973 , piece_id=VJLEMOZ5URRVHV2YCUKCIXV3JDHF3FAK32SEDBFLPURLLRSBARHA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.748562254 (3h42m28.788135374s) [noise=true , address=45.91.116.9:4262 , piece_id=5ETIFQYUABHYAD23OABLKMRONKP3FAMOFF4HSRNYNCKYMGQC3MBA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749067632 (3h42m28.788640752s) [noise=true , address=77.90.31.12:4589 , piece_id=RJYFTKEQTI46J5J2VHITLMMOYRH7TSBLBAEH3NZQADQZH7VPJEUA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749075707 (3h42m28.788648817s) [noise=true , address=45.91.116.11:4117 , piece_id=BRRXVJ4ABS762RVT4GEBQPHHDREZRYJDFKNR4WURP55IRBFHVGRA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749355202 (3h42m28.788928322s) [noise=true , address=45.91.116.12:4226 , piece_id=MKGMT47CVESBJPSMQAE7LQ4RNALF5BOQOTVZRFFFFO5Z66EYQLBQ, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749388265 (3h42m28.788961385s) [noise=true , address=45.91.116.4:4938 , piece_id=RYTEHGP6AFZENL7EPTTAGOXJAFIN5S5VQNINZFSK3QK3OLEX6MMA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749663221 (3h42m28.789236341s) [noise=true , address=77.90.31.19:4843 , piece_id=3HMXQELRZRVS5DQ7WW7LIVMCJSQKCDOXKZAWRN5HF2EUZMKTEHXA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749708768 (3h42m28.789281888s) [noise=true , address=45.91.116.7:4010 , piece_id=TGYLFVVW2FQEV3NEVQY5BHLXKM3SJO2GRJYIBIEV5I6CJOERZ34Q, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749967193 (3h42m28.789540313s) [noise=true , address=45.91.116.9:4263 , piece_id=NEZSPOFVK6Z4AOIVRNTINFZ7GDU3IGDTKSTA6O4SBXM7P6QJIJBA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.750066543 (3h42m28.789639663s) [noise=true , address=45.91.116.4:4940 , piece_id=HSXTCVLUG65EQYQNQIU7IFBU4NM2ZBSGTCKBP5PQDOINJZLZ6SRA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.750338233 (3h42m28.789911353s) [noise=true , address=45.91.116.5:4298 , piece_id=JBUC2FAL7DLRRBG5D2MQOSRZ7I4CUA2VFUK6IVFNFNPOT643YSRQ, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.750606877 (3h42m28.790179997s) [noise=true , address=45.91.116.8:4047 , piece_id=3D5N72MAOC62Y7DAEGBF3NXZTCCVKKNLEJM4JH2IQGYKYBFLZGOA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.750851094 (3h42m28.790424214s) [noise=true , address=45.91.116.5:4297 , piece_id=CRXAS52BRF5BWIRTIUCSSJ6H2EGFLMCFRBTCDHGZDMZI4OY5QPAA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.751047630 (3h42m28.79062075s) [noise=true , address=45.91.116.6:4974 , piece_id=QCOOIZ4IYM5EQ6AXV6DZVUYPU3YL75J7S5DWJZKSOA4YVRG6IJEQ, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.752127818 (3h42m28.791700938s) [noise=true , address=45.91.116.7:4009 , piece_id=HTY4BZ3H7UTXHEWYS2BNQKWVOIS4M3LH5ERNX6HVNBJXG7M7XNYQ, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.752356635 (3h42m28.791929755s) [noise=true , address=77.90.31.17:4771 , piece_id=KQJDCQYHVTQLLMURKEMN452LRAO4OHFVSWZYXSB3K5KSGNRNCGPA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:54.590291704 (3h42m30.629864824s) [upload=3230]: Next returned nil segment Sep 6 17:10:54.590312094 (3h42m30.629885204s) [upload=3230]: Waiting for error group managing segments... Sep 6 17:10:54.929472770 (3h42m30.96904589s) [noise=true , address=45.91.116.4:4938 , piece_id=RYTEHGP6AFZENL7EPTTAGOXJAFIN5S5VQNINZFSK3QK3OLEX6MMA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:54.938832018 (3h42m30.978405138s) [noise=true , address=45.91.116.4:4940 , piece_id=HSXTCVLUG65EQYQNQIU7IFBU4NM2ZBSGTCKBP5PQDOINJZLZ6SRA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:54.960049478 (3h42m30.999622598s) [noise=true , address=45.91.116.12:4226 , piece_id=MKGMT47CVESBJPSMQAE7LQ4RNALF5BOQOTVZRFFFFO5Z66EYQLBQ, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:54.966761922 (3h42m31.006335042s) [noise=true , address=45.91.116.9:4262 , piece_id=5ETIFQYUABHYAD23OABLKMRONKP3FAMOFF4HSRNYNCKYMGQC3MBA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:54.970878660 (3h42m31.01045178s) [noise=true , address=45.91.116.6:4974 , piece_id=QCOOIZ4IYM5EQ6AXV6DZVUYPU3YL75J7S5DWJZKSOA4YVRG6IJEQ, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.001147149 (3h42m31.040720269s) [noise=true , address=45.91.116.7:4010 , piece_id=TGYLFVVW2FQEV3NEVQY5BHLXKM3SJO2GRJYIBIEV5I6CJOERZ34Q, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.009058858 (3h42m31.048631978s) [noise=true , address=45.91.116.9:4263 , piece_id=NEZSPOFVK6Z4AOIVRNTINFZ7GDU3IGDTKSTA6O4SBXM7P6QJIJBA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.028713858 (3h42m31.068286978s) [noise=true , address=77.90.31.19:4843 , piece_id=3HMXQELRZRVS5DQ7WW7LIVMCJSQKCDOXKZAWRN5HF2EUZMKTEHXA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.086371005 (3h42m31.125944125s) [noise=true , address=45.91.116.6:4973 , piece_id=VJLEMOZ5URRVHV2YCUKCIXV3JDHF3FAK32SEDBFLPURLLRSBARHA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.097504971 (3h42m31.137078091s) [noise=true , address=45.91.116.8:4047 , piece_id=3D5N72MAOC62Y7DAEGBF3NXZTCCVKKNLEJM4JH2IQGYKYBFLZGOA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.131107198 (3h42m31.170680318s) [noise=true , address=77.90.31.17:4771 , piece_id=KQJDCQYHVTQLLMURKEMN452LRAO4OHFVSWZYXSB3K5KSGNRNCGPA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.173086245 (3h42m31.212659385s) [noise=true , address=45.91.116.5:4298 , piece_id=JBUC2FAL7DLRRBG5D2MQOSRZ7I4CUA2VFUK6IVFNFNPOT643YSRQ, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.188757552 (3h42m31.228330672s) [noise=true , address=45.91.116.7:4009 , piece_id=HTY4BZ3H7UTXHEWYS2BNQKWVOIS4M3LH5ERNX6HVNBJXG7M7XNYQ, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.188782540 (3h42m31.22835566s) [seg_pos={0 0}, upload=3230]: Segment reached optimal threshold of 13 pieces. Sep 6 17:10:55.188896879 (3h42m31.228469999s) [noise=true , address=45.91.116.5:4297 , piece_id=CRXAS52BRF5BWIRTIUCSSJ6H2EGFLMCFRBTCDHGZDMZI4OY5QPAA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Sep 6 17:10:55.188948438 (3h42m31.228521558s) [noise=true , address=45.91.116.11:4117 , piece_id=BRRXVJ4ABS762RVT4GEBQPHHDREZRYJDFKNR4WURP55IRBFHVGRA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 6 17:10:55.189010556 (3h42m31.228583676s) [noise=true , address=77.90.31.12:4589 , piece_id=RJYFTKEQTI46J5J2VHITLMMOYRH7TSBLBAEH3NZQADQZH7VPJEUA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: write tcp 188.166.225.170:53128->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:53128->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53128->77.90.31.12:4589: use of closed network connection Sep 6 17:10:55.189021337 (3h42m31.228594457s) [seg_pos={0 0}, upload=3230]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 17:10:55.189082755 (3h42m31.228655865s) [upload=3230]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 17:10:55.189086422 (3h42m31.228659542s) [upload=3230]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 17:10:55.698217639 (3h42m31.737790759s) [upload=3230]: Done uploading segments... Sep 6 17:10:55.698268134 (3h42m31.737841255s) [upload=3230]: Upload finished. err: <nil> Duration: 6.900193955s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 24 out of 60 is failed (threshold=3) [XX..X.X.XX....XX.XXX.....X..X....X......X.XX.X....XX..XXX..X] duration: 30.002270878s FAIL (2025-09-06 17:11:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 17:10:52.470391152 (3h42m28.509964272s) [upload=3230]: Starting upload Sep 6 17:10:52.470445657 (3h42m28.510018777s) [upload=3230]: Done starting upload Sep 6 17:10:52.470620361 (3h42m28.510193481s) [upload=3230]: Uploading segments... Sep 6 17:10:52.470656090 (3h42m28.51022921s) [upload=3230]: Got next segment. Inline: false Sep 6 17:10:52.470678181 (3h42m28.510251301s) [upload=3230]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 17:10:52.470682490 (3h42m28.5102556s) [upload=3230]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 17:10:52.748377991 (3h42m28.787951152s) [seg_pos={0 0}, upload=3230]: Begin upload segment... Sep 6 17:10:52.748441253 (3h42m28.788014393s) [seg_pos={0 0}, upload=3230]: Done begin upload segment. Sep 6 17:10:52.748513080 (3h42m28.7880862s) [noise=true , address=45.91.116.6:4973 , piece_id=VJLEMOZ5URRVHV2YCUKCIXV3JDHF3FAK32SEDBFLPURLLRSBARHA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.748562254 (3h42m28.788135374s) [noise=true , address=45.91.116.9:4262 , piece_id=5ETIFQYUABHYAD23OABLKMRONKP3FAMOFF4HSRNYNCKYMGQC3MBA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749067632 (3h42m28.788640752s) [noise=true , address=77.90.31.12:4589 , piece_id=RJYFTKEQTI46J5J2VHITLMMOYRH7TSBLBAEH3NZQADQZH7VPJEUA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749075707 (3h42m28.788648817s) [noise=true , address=45.91.116.11:4117 , piece_id=BRRXVJ4ABS762RVT4GEBQPHHDREZRYJDFKNR4WURP55IRBFHVGRA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749355202 (3h42m28.788928322s) [noise=true , address=45.91.116.12:4226 , piece_id=MKGMT47CVESBJPSMQAE7LQ4RNALF5BOQOTVZRFFFFO5Z66EYQLBQ, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749388265 (3h42m28.788961385s) [noise=true , address=45.91.116.4:4938 , piece_id=RYTEHGP6AFZENL7EPTTAGOXJAFIN5S5VQNINZFSK3QK3OLEX6MMA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749663221 (3h42m28.789236341s) [noise=true , address=77.90.31.19:4843 , piece_id=3HMXQELRZRVS5DQ7WW7LIVMCJSQKCDOXKZAWRN5HF2EUZMKTEHXA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749708768 (3h42m28.789281888s) [noise=true , address=45.91.116.7:4010 , piece_id=TGYLFVVW2FQEV3NEVQY5BHLXKM3SJO2GRJYIBIEV5I6CJOERZ34Q, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.749967193 (3h42m28.789540313s) [noise=true , address=45.91.116.9:4263 , piece_id=NEZSPOFVK6Z4AOIVRNTINFZ7GDU3IGDTKSTA6O4SBXM7P6QJIJBA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.750066543 (3h42m28.789639663s) [noise=true , address=45.91.116.4:4940 , piece_id=HSXTCVLUG65EQYQNQIU7IFBU4NM2ZBSGTCKBP5PQDOINJZLZ6SRA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.750338233 (3h42m28.789911353s) [noise=true , address=45.91.116.5:4298 , piece_id=JBUC2FAL7DLRRBG5D2MQOSRZ7I4CUA2VFUK6IVFNFNPOT643YSRQ, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.750606877 (3h42m28.790179997s) [noise=true , address=45.91.116.8:4047 , piece_id=3D5N72MAOC62Y7DAEGBF3NXZTCCVKKNLEJM4JH2IQGYKYBFLZGOA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.750851094 (3h42m28.790424214s) [noise=true , address=45.91.116.5:4297 , piece_id=CRXAS52BRF5BWIRTIUCSSJ6H2EGFLMCFRBTCDHGZDMZI4OY5QPAA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.751047630 (3h42m28.79062075s) [noise=true , address=45.91.116.6:4974 , piece_id=QCOOIZ4IYM5EQ6AXV6DZVUYPU3YL75J7S5DWJZKSOA4YVRG6IJEQ, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.752127818 (3h42m28.791700938s) [noise=true , address=45.91.116.7:4009 , piece_id=HTY4BZ3H7UTXHEWYS2BNQKWVOIS4M3LH5ERNX6HVNBJXG7M7XNYQ, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:52.752356635 (3h42m28.791929755s) [noise=true , address=77.90.31.17:4771 , piece_id=KQJDCQYHVTQLLMURKEMN452LRAO4OHFVSWZYXSB3K5KSGNRNCGPA, seg_pos={0 0}, upload=3230]: Uploading piece... Sep 6 17:10:54.590291704 (3h42m30.629864824s) [upload=3230]: Next returned nil segment Sep 6 17:10:54.590312094 (3h42m30.629885204s) [upload=3230]: Waiting for error group managing segments... Sep 6 17:10:54.929472770 (3h42m30.96904589s) [noise=true , address=45.91.116.4:4938 , piece_id=RYTEHGP6AFZENL7EPTTAGOXJAFIN5S5VQNINZFSK3QK3OLEX6MMA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:54.938832018 (3h42m30.978405138s) [noise=true , address=45.91.116.4:4940 , piece_id=HSXTCVLUG65EQYQNQIU7IFBU4NM2ZBSGTCKBP5PQDOINJZLZ6SRA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:54.960049478 (3h42m30.999622598s) [noise=true , address=45.91.116.12:4226 , piece_id=MKGMT47CVESBJPSMQAE7LQ4RNALF5BOQOTVZRFFFFO5Z66EYQLBQ, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:54.966761922 (3h42m31.006335042s) [noise=true , address=45.91.116.9:4262 , piece_id=5ETIFQYUABHYAD23OABLKMRONKP3FAMOFF4HSRNYNCKYMGQC3MBA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:54.970878660 (3h42m31.01045178s) [noise=true , address=45.91.116.6:4974 , piece_id=QCOOIZ4IYM5EQ6AXV6DZVUYPU3YL75J7S5DWJZKSOA4YVRG6IJEQ, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.001147149 (3h42m31.040720269s) [noise=true , address=45.91.116.7:4010 , piece_id=TGYLFVVW2FQEV3NEVQY5BHLXKM3SJO2GRJYIBIEV5I6CJOERZ34Q, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.009058858 (3h42m31.048631978s) [noise=true , address=45.91.116.9:4263 , piece_id=NEZSPOFVK6Z4AOIVRNTINFZ7GDU3IGDTKSTA6O4SBXM7P6QJIJBA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.028713858 (3h42m31.068286978s) [noise=true , address=77.90.31.19:4843 , piece_id=3HMXQELRZRVS5DQ7WW7LIVMCJSQKCDOXKZAWRN5HF2EUZMKTEHXA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.086371005 (3h42m31.125944125s) [noise=true , address=45.91.116.6:4973 , piece_id=VJLEMOZ5URRVHV2YCUKCIXV3JDHF3FAK32SEDBFLPURLLRSBARHA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.097504971 (3h42m31.137078091s) [noise=true , address=45.91.116.8:4047 , piece_id=3D5N72MAOC62Y7DAEGBF3NXZTCCVKKNLEJM4JH2IQGYKYBFLZGOA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.131107198 (3h42m31.170680318s) [noise=true , address=77.90.31.17:4771 , piece_id=KQJDCQYHVTQLLMURKEMN452LRAO4OHFVSWZYXSB3K5KSGNRNCGPA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.173086245 (3h42m31.212659385s) [noise=true , address=45.91.116.5:4298 , piece_id=JBUC2FAL7DLRRBG5D2MQOSRZ7I4CUA2VFUK6IVFNFNPOT643YSRQ, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.188757552 (3h42m31.228330672s) [noise=true , address=45.91.116.7:4009 , piece_id=HTY4BZ3H7UTXHEWYS2BNQKWVOIS4M3LH5ERNX6HVNBJXG7M7XNYQ, seg_pos={0 0}, upload=3230]: Done uploading piece. err: <nil> Sep 6 17:10:55.188782540 (3h42m31.22835566s) [seg_pos={0 0}, upload=3230]: Segment reached optimal threshold of 13 pieces. Sep 6 17:10:55.188896879 (3h42m31.228469999s) [noise=true , address=45.91.116.5:4297 , piece_id=CRXAS52BRF5BWIRTIUCSSJ6H2EGFLMCFRBTCDHGZDMZI4OY5QPAA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: expected piece hash; context canceled; EOF Sep 6 17:10:55.188948438 (3h42m31.228521558s) [noise=true , address=45.91.116.11:4117 , piece_id=BRRXVJ4ABS762RVT4GEBQPHHDREZRYJDFKNR4WURP55IRBFHVGRA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 6 17:10:55.189010556 (3h42m31.228583676s) [noise=true , address=77.90.31.12:4589 , piece_id=RJYFTKEQTI46J5J2VHITLMMOYRH7TSBLBAEH3NZQADQZH7VPJEUA, seg_pos={0 0}, upload=3230]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: write tcp 188.166.225.170:53128->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:53128->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53128->77.90.31.12:4589: use of closed network connection Sep 6 17:10:55.189021337 (3h42m31.228594457s) [seg_pos={0 0}, upload=3230]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 17:10:55.189082755 (3h42m31.228655865s) [upload=3230]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 17:10:55.189086422 (3h42m31.228659542s) [upload=3230]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 17:10:55.698217639 (3h42m31.737790759s) [upload=3230]: Done uploading segments... Sep 6 17:10:55.698268134 (3h42m31.737841255s) [upload=3230]: Upload finished. err: <nil> Duration: 30.002411197s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [.XX..X.X.XX....XX.XXX.....X..X....X......X.XX.X....XX..XXX..] duration: 22.314790119s FAIL (2025-09-06 17:10:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 17:08:26.687712680 (3h40m2.7272858s) [upload=3196]: Starting upload Sep 6 17:08:26.687783666 (3h40m2.727356786s) [upload=3196]: Done starting upload Sep 6 17:08:26.687893336 (3h40m2.727466446s) [upload=3196]: Uploading segments... Sep 6 17:08:26.688013836 (3h40m2.727586956s) [upload=3196]: Got next segment. Inline: false Sep 6 17:08:26.688041028 (3h40m2.727614148s) [upload=3196]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 17:08:26.688045516 (3h40m2.727618636s) [upload=3196]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 17:08:26.960540468 (3h40m3.000113578s) [seg_pos={0 0}, upload=3196]: Begin upload segment... Sep 6 17:08:26.960577538 (3h40m3.000150648s) [seg_pos={0 0}, upload=3196]: Done begin upload segment. Sep 6 17:08:26.960602276 (3h40m3.000175396s) [noise=true , address=77.90.31.10:4517 , piece_id=YCYEXOGV2V4JBRJO7UCKJP5ALBJHXWBGHR7Z6KO6CFJQJMH4ZMRA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.960616103 (3h40m3.000189223s) [noise=true , address=45.91.116.5:4299 , piece_id=SFQ4OFVFFEOCCWUEPYHSJT227DOYNG43NI2FHEKYLM2PTWLJWFSA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.960968116 (3h40m3.000541236s) [noise=true , address=45.91.116.4:4937 , piece_id=NUHGSVRL4NY23PDRLNRIDR7BBDJNY6X2VML2LBKNMRVNKFMMRMNA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.960993926 (3h40m3.000567126s) [noise=true , address=77.90.31.10:4520 , piece_id=X44SZGRTPHFHX24FDFBVBKRMD4RCZP4A6PN7SAYLQT5GIV4MRLQA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.961244545 (3h40m3.000817665s) [noise=true , address=77.90.31.16:4733 , piece_id=BSSUJWNUOAUUVJ6PBDUAY2VQPE3YAMQXXSBFWLWPI6QUNJAQXQWA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.961273791 (3h40m3.000846911s) [noise=true , address=45.91.116.9:4261 , piece_id=6G2NRF52FE2K3PALYEC3CANKRHO7NDTPHV733KTI5O7SQDDZXNWA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.961538588 (3h40m3.001111698s) [noise=true , address=45.91.116.11:4119 , piece_id=4QAHNDRRU2CYUQFVWUZ6QX6XDPXISDWOS2HPSGXY44MGZHA42T4A, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.961562423 (3h40m3.001135543s) [noise=true , address=45.91.116.10:4083 , piece_id=XO6E7AKAP2LVD4A4JOCP5TCBPZVATB4RMJXEMO4ST4EIBJZYVSWQ, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.961825236 (3h40m3.001398356s) [noise=true , address=45.91.116.7:4009 , piece_id=X2TGDVQGKFFJVM5UPKQ5TL7O76VQ7F3YTK3ZXD7F6NWCIXFRELCA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.961871174 (3h40m3.001444294s) [noise=true , address=45.91.116.13:4155 , piece_id=KVSLFHZ527LMCC4I2RB7SABZYMK4YQGA57K6D67VG254SOCTUMSQ, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.962043073 (3h40m3.001616193s) [noise=true , address=45.91.116.6:4975 , piece_id=MI5EIXGJSHALBJQCYJWJHQSDMHELUZJKUPNL4RUXZRZ6LWHKYSPA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.962093670 (3h40m3.0016668s) [noise=true , address=77.90.31.12:4592 , piece_id=E76LXWQJAMCYRZQ2B2ZTFNLZGNOOOOB7ADZMG4ZUC272WXUCAYNQ, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.962253967 (3h40m3.001827087s) [noise=true , address=45.91.116.7:4011 , piece_id=UTCBQCPJF7AVXKCCO5WFRWBKNI6PJZGOJS2JIAU5NQA3PPRFTDQQ, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.962541326 (3h40m3.002114446s) [noise=true , address=77.90.31.15:4697 , piece_id=XNWIQLO6WUMEDA6J6GZNMTDKJ5CAIMJI2EAJUDGXZ3JLYTCA5NEA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.962760135 (3h40m3.002333255s) [noise=true , address=45.91.116.8:4048 , piece_id=VU3QV7VQEWDFGRNLCYGYVFOOJG4RL4XQRZI7DASTUMZIH24DSQUA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.962949517 (3h40m3.002522637s) [noise=true , address=77.90.31.21:4879 , piece_id=ZO7ECYWFHRCD7XCZ6EOS6OH2NG3DTK2HHGIFQXUDQWKQZ5WY2CXA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:28.782128508 (3h40m4.821701628s) [upload=3196]: Next returned nil segment Sep 6 17:08:28.782146862 (3h40m4.821719982s) [upload=3196]: Waiting for error group managing segments... Sep 6 17:08:29.086638188 (3h40m5.126211308s) [noise=true , address=77.90.31.10:4520 , piece_id=X44SZGRTPHFHX24FDFBVBKRMD4RCZP4A6PN7SAYLQT5GIV4MRLQA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.118591722 (3h40m5.158164842s) [noise=true , address=45.91.116.13:4155 , piece_id=KVSLFHZ527LMCC4I2RB7SABZYMK4YQGA57K6D67VG254SOCTUMSQ, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.131657513 (3h40m5.171230633s) [noise=true , address=45.91.116.5:4299 , piece_id=SFQ4OFVFFEOCCWUEPYHSJT227DOYNG43NI2FHEKYLM2PTWLJWFSA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.141747511 (3h40m5.181320661s) [noise=true , address=45.91.116.7:4009 , piece_id=X2TGDVQGKFFJVM5UPKQ5TL7O76VQ7F3YTK3ZXD7F6NWCIXFRELCA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.180750593 (3h40m5.220323713s) [noise=true , address=45.91.116.6:4975 , piece_id=MI5EIXGJSHALBJQCYJWJHQSDMHELUZJKUPNL4RUXZRZ6LWHKYSPA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.207144909 (3h40m5.246718019s) [noise=true , address=45.91.116.11:4119 , piece_id=4QAHNDRRU2CYUQFVWUZ6QX6XDPXISDWOS2HPSGXY44MGZHA42T4A, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.216984887 (3h40m5.256558007s) [noise=true , address=45.91.116.8:4048 , piece_id=VU3QV7VQEWDFGRNLCYGYVFOOJG4RL4XQRZI7DASTUMZIH24DSQUA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.322551711 (3h40m5.362124821s) [noise=true , address=45.91.116.9:4261 , piece_id=6G2NRF52FE2K3PALYEC3CANKRHO7NDTPHV733KTI5O7SQDDZXNWA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.432594197 (3h40m5.472167317s) [noise=true , address=45.91.116.4:4937 , piece_id=NUHGSVRL4NY23PDRLNRIDR7BBDJNY6X2VML2LBKNMRVNKFMMRMNA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.438842063 (3h40m5.478415183s) [noise=true , address=45.91.116.7:4011 , piece_id=UTCBQCPJF7AVXKCCO5WFRWBKNI6PJZGOJS2JIAU5NQA3PPRFTDQQ, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.476747214 (3h40m5.516320334s) [noise=true , address=77.90.31.12:4592 , piece_id=E76LXWQJAMCYRZQ2B2ZTFNLZGNOOOOB7ADZMG4ZUC272WXUCAYNQ, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.531325830 (3h40m5.57089895s) [noise=true , address=77.90.31.21:4879 , piece_id=ZO7ECYWFHRCD7XCZ6EOS6OH2NG3DTK2HHGIFQXUDQWKQZ5WY2CXA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.619125606 (3h40m5.658698726s) [noise=true , address=77.90.31.10:4517 , piece_id=YCYEXOGV2V4JBRJO7UCKJP5ALBJHXWBGHR7Z6KO6CFJQJMH4ZMRA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.619153470 (3h40m5.65872659s) [seg_pos={0 0}, upload=3196]: Segment reached optimal threshold of 13 pieces. Sep 6 17:08:29.619271535 (3h40m5.658844645s) [noise=true , address=77.90.31.15:4697 , piece_id=XNWIQLO6WUMEDA6J6GZNMTDKJ5CAIMJI2EAJUDGXZ3JLYTCA5NEA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5): protocol: expected piece hash; context canceled; EOF Sep 6 17:08:29.619458623 (3h40m5.659031743s) [noise=true , address=77.90.31.16:4733 , piece_id=BSSUJWNUOAUUVJ6PBDUAY2VQPE3YAMQXXSBFWLWPI6QUNJAQXQWA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:51836->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:51836->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51836->77.90.31.16:4733: use of closed network connection Sep 6 17:08:29.619495864 (3h40m5.659068974s) [noise=true , address=45.91.116.10:4083 , piece_id=XO6E7AKAP2LVD4A4JOCP5TCBPZVATB4RMJXEMO4ST4EIBJZYVSWQ, seg_pos={0 0}, upload=3196]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:57014->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:57014->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57014->45.91.116.10:4083: use of closed network connection Sep 6 17:08:29.619506666 (3h40m5.659079786s) [seg_pos={0 0}, upload=3196]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 17:08:29.619565368 (3h40m5.659138488s) [upload=3196]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 17:08:29.619569155 (3h40m5.659142275s) [upload=3196]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 17:08:29.981128051 (3h40m6.020701171s) [upload=3196]: Done uploading segments... Sep 6 17:08:29.981146546 (3h40m6.020719666s) [upload=3196]: Upload finished. err: <nil> Duration: 22.314810489s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [..XX..X.X.XX....XX.XXX.....X..X....X......X.XX.X....XX..XXX.] duration: 20.359653426s FAIL (2025-09-06 17:09:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 17:08:26.687712680 (3h40m2.7272858s) [upload=3196]: Starting upload Sep 6 17:08:26.687783666 (3h40m2.727356786s) [upload=3196]: Done starting upload Sep 6 17:08:26.687893336 (3h40m2.727466446s) [upload=3196]: Uploading segments... Sep 6 17:08:26.688013836 (3h40m2.727586956s) [upload=3196]: Got next segment. Inline: false Sep 6 17:08:26.688041028 (3h40m2.727614148s) [upload=3196]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 17:08:26.688045516 (3h40m2.727618636s) [upload=3196]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 17:08:26.960540468 (3h40m3.000113578s) [seg_pos={0 0}, upload=3196]: Begin upload segment... Sep 6 17:08:26.960577538 (3h40m3.000150648s) [seg_pos={0 0}, upload=3196]: Done begin upload segment. Sep 6 17:08:26.960602276 (3h40m3.000175396s) [noise=true , address=77.90.31.10:4517 , piece_id=YCYEXOGV2V4JBRJO7UCKJP5ALBJHXWBGHR7Z6KO6CFJQJMH4ZMRA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.960616103 (3h40m3.000189223s) [noise=true , address=45.91.116.5:4299 , piece_id=SFQ4OFVFFEOCCWUEPYHSJT227DOYNG43NI2FHEKYLM2PTWLJWFSA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.960968116 (3h40m3.000541236s) [noise=true , address=45.91.116.4:4937 , piece_id=NUHGSVRL4NY23PDRLNRIDR7BBDJNY6X2VML2LBKNMRVNKFMMRMNA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.960993926 (3h40m3.000567126s) [noise=true , address=77.90.31.10:4520 , piece_id=X44SZGRTPHFHX24FDFBVBKRMD4RCZP4A6PN7SAYLQT5GIV4MRLQA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.961244545 (3h40m3.000817665s) [noise=true , address=77.90.31.16:4733 , piece_id=BSSUJWNUOAUUVJ6PBDUAY2VQPE3YAMQXXSBFWLWPI6QUNJAQXQWA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.961273791 (3h40m3.000846911s) [noise=true , address=45.91.116.9:4261 , piece_id=6G2NRF52FE2K3PALYEC3CANKRHO7NDTPHV733KTI5O7SQDDZXNWA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.961538588 (3h40m3.001111698s) [noise=true , address=45.91.116.11:4119 , piece_id=4QAHNDRRU2CYUQFVWUZ6QX6XDPXISDWOS2HPSGXY44MGZHA42T4A, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.961562423 (3h40m3.001135543s) [noise=true , address=45.91.116.10:4083 , piece_id=XO6E7AKAP2LVD4A4JOCP5TCBPZVATB4RMJXEMO4ST4EIBJZYVSWQ, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.961825236 (3h40m3.001398356s) [noise=true , address=45.91.116.7:4009 , piece_id=X2TGDVQGKFFJVM5UPKQ5TL7O76VQ7F3YTK3ZXD7F6NWCIXFRELCA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.961871174 (3h40m3.001444294s) [noise=true , address=45.91.116.13:4155 , piece_id=KVSLFHZ527LMCC4I2RB7SABZYMK4YQGA57K6D67VG254SOCTUMSQ, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.962043073 (3h40m3.001616193s) [noise=true , address=45.91.116.6:4975 , piece_id=MI5EIXGJSHALBJQCYJWJHQSDMHELUZJKUPNL4RUXZRZ6LWHKYSPA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.962093670 (3h40m3.0016668s) [noise=true , address=77.90.31.12:4592 , piece_id=E76LXWQJAMCYRZQ2B2ZTFNLZGNOOOOB7ADZMG4ZUC272WXUCAYNQ, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.962253967 (3h40m3.001827087s) [noise=true , address=45.91.116.7:4011 , piece_id=UTCBQCPJF7AVXKCCO5WFRWBKNI6PJZGOJS2JIAU5NQA3PPRFTDQQ, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.962541326 (3h40m3.002114446s) [noise=true , address=77.90.31.15:4697 , piece_id=XNWIQLO6WUMEDA6J6GZNMTDKJ5CAIMJI2EAJUDGXZ3JLYTCA5NEA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.962760135 (3h40m3.002333255s) [noise=true , address=45.91.116.8:4048 , piece_id=VU3QV7VQEWDFGRNLCYGYVFOOJG4RL4XQRZI7DASTUMZIH24DSQUA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.962949517 (3h40m3.002522637s) [noise=true , address=77.90.31.21:4879 , piece_id=ZO7ECYWFHRCD7XCZ6EOS6OH2NG3DTK2HHGIFQXUDQWKQZ5WY2CXA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:28.782128508 (3h40m4.821701628s) [upload=3196]: Next returned nil segment Sep 6 17:08:28.782146862 (3h40m4.821719982s) [upload=3196]: Waiting for error group managing segments... Sep 6 17:08:29.086638188 (3h40m5.126211308s) [noise=true , address=77.90.31.10:4520 , piece_id=X44SZGRTPHFHX24FDFBVBKRMD4RCZP4A6PN7SAYLQT5GIV4MRLQA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.118591722 (3h40m5.158164842s) [noise=true , address=45.91.116.13:4155 , piece_id=KVSLFHZ527LMCC4I2RB7SABZYMK4YQGA57K6D67VG254SOCTUMSQ, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.131657513 (3h40m5.171230633s) [noise=true , address=45.91.116.5:4299 , piece_id=SFQ4OFVFFEOCCWUEPYHSJT227DOYNG43NI2FHEKYLM2PTWLJWFSA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.141747511 (3h40m5.181320661s) [noise=true , address=45.91.116.7:4009 , piece_id=X2TGDVQGKFFJVM5UPKQ5TL7O76VQ7F3YTK3ZXD7F6NWCIXFRELCA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.180750593 (3h40m5.220323713s) [noise=true , address=45.91.116.6:4975 , piece_id=MI5EIXGJSHALBJQCYJWJHQSDMHELUZJKUPNL4RUXZRZ6LWHKYSPA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.207144909 (3h40m5.246718019s) [noise=true , address=45.91.116.11:4119 , piece_id=4QAHNDRRU2CYUQFVWUZ6QX6XDPXISDWOS2HPSGXY44MGZHA42T4A, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.216984887 (3h40m5.256558007s) [noise=true , address=45.91.116.8:4048 , piece_id=VU3QV7VQEWDFGRNLCYGYVFOOJG4RL4XQRZI7DASTUMZIH24DSQUA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.322551711 (3h40m5.362124821s) [noise=true , address=45.91.116.9:4261 , piece_id=6G2NRF52FE2K3PALYEC3CANKRHO7NDTPHV733KTI5O7SQDDZXNWA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.432594197 (3h40m5.472167317s) [noise=true , address=45.91.116.4:4937 , piece_id=NUHGSVRL4NY23PDRLNRIDR7BBDJNY6X2VML2LBKNMRVNKFMMRMNA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.438842063 (3h40m5.478415183s) [noise=true , address=45.91.116.7:4011 , piece_id=UTCBQCPJF7AVXKCCO5WFRWBKNI6PJZGOJS2JIAU5NQA3PPRFTDQQ, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.476747214 (3h40m5.516320334s) [noise=true , address=77.90.31.12:4592 , piece_id=E76LXWQJAMCYRZQ2B2ZTFNLZGNOOOOB7ADZMG4ZUC272WXUCAYNQ, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.531325830 (3h40m5.57089895s) [noise=true , address=77.90.31.21:4879 , piece_id=ZO7ECYWFHRCD7XCZ6EOS6OH2NG3DTK2HHGIFQXUDQWKQZ5WY2CXA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.619125606 (3h40m5.658698726s) [noise=true , address=77.90.31.10:4517 , piece_id=YCYEXOGV2V4JBRJO7UCKJP5ALBJHXWBGHR7Z6KO6CFJQJMH4ZMRA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.619153470 (3h40m5.65872659s) [seg_pos={0 0}, upload=3196]: Segment reached optimal threshold of 13 pieces. Sep 6 17:08:29.619271535 (3h40m5.658844645s) [noise=true , address=77.90.31.15:4697 , piece_id=XNWIQLO6WUMEDA6J6GZNMTDKJ5CAIMJI2EAJUDGXZ3JLYTCA5NEA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5): protocol: expected piece hash; context canceled; EOF Sep 6 17:08:29.619458623 (3h40m5.659031743s) [noise=true , address=77.90.31.16:4733 , piece_id=BSSUJWNUOAUUVJ6PBDUAY2VQPE3YAMQXXSBFWLWPI6QUNJAQXQWA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:51836->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:51836->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51836->77.90.31.16:4733: use of closed network connection Sep 6 17:08:29.619495864 (3h40m5.659068974s) [noise=true , address=45.91.116.10:4083 , piece_id=XO6E7AKAP2LVD4A4JOCP5TCBPZVATB4RMJXEMO4ST4EIBJZYVSWQ, seg_pos={0 0}, upload=3196]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:57014->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:57014->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57014->45.91.116.10:4083: use of closed network connection Sep 6 17:08:29.619506666 (3h40m5.659079786s) [seg_pos={0 0}, upload=3196]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 17:08:29.619565368 (3h40m5.659138488s) [upload=3196]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 17:08:29.619569155 (3h40m5.659142275s) [upload=3196]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 17:08:29.981128051 (3h40m6.020701171s) [upload=3196]: Done uploading segments... Sep 6 17:08:29.981146546 (3h40m6.020719666s) [upload=3196]: Upload finished. err: <nil> Duration: 20.359679084s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 24 out of 60 is failed (threshold=3) [X..XX..X.X.XX....XX.XXX.....X..X....X......X.XX.X....XX..XXX] duration: 30.002072904s FAIL (2025-09-06 17:08:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 17:08:26.687712680 (3h40m2.7272858s) [upload=3196]: Starting upload Sep 6 17:08:26.687783666 (3h40m2.727356786s) [upload=3196]: Done starting upload Sep 6 17:08:26.687893336 (3h40m2.727466446s) [upload=3196]: Uploading segments... Sep 6 17:08:26.688013836 (3h40m2.727586956s) [upload=3196]: Got next segment. Inline: false Sep 6 17:08:26.688041028 (3h40m2.727614148s) [upload=3196]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 17:08:26.688045516 (3h40m2.727618636s) [upload=3196]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 17:08:26.960540468 (3h40m3.000113578s) [seg_pos={0 0}, upload=3196]: Begin upload segment... Sep 6 17:08:26.960577538 (3h40m3.000150648s) [seg_pos={0 0}, upload=3196]: Done begin upload segment. Sep 6 17:08:26.960602276 (3h40m3.000175396s) [noise=true , address=77.90.31.10:4517 , piece_id=YCYEXOGV2V4JBRJO7UCKJP5ALBJHXWBGHR7Z6KO6CFJQJMH4ZMRA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.960616103 (3h40m3.000189223s) [noise=true , address=45.91.116.5:4299 , piece_id=SFQ4OFVFFEOCCWUEPYHSJT227DOYNG43NI2FHEKYLM2PTWLJWFSA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.960968116 (3h40m3.000541236s) [noise=true , address=45.91.116.4:4937 , piece_id=NUHGSVRL4NY23PDRLNRIDR7BBDJNY6X2VML2LBKNMRVNKFMMRMNA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.960993926 (3h40m3.000567126s) [noise=true , address=77.90.31.10:4520 , piece_id=X44SZGRTPHFHX24FDFBVBKRMD4RCZP4A6PN7SAYLQT5GIV4MRLQA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.961244545 (3h40m3.000817665s) [noise=true , address=77.90.31.16:4733 , piece_id=BSSUJWNUOAUUVJ6PBDUAY2VQPE3YAMQXXSBFWLWPI6QUNJAQXQWA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.961273791 (3h40m3.000846911s) [noise=true , address=45.91.116.9:4261 , piece_id=6G2NRF52FE2K3PALYEC3CANKRHO7NDTPHV733KTI5O7SQDDZXNWA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.961538588 (3h40m3.001111698s) [noise=true , address=45.91.116.11:4119 , piece_id=4QAHNDRRU2CYUQFVWUZ6QX6XDPXISDWOS2HPSGXY44MGZHA42T4A, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.961562423 (3h40m3.001135543s) [noise=true , address=45.91.116.10:4083 , piece_id=XO6E7AKAP2LVD4A4JOCP5TCBPZVATB4RMJXEMO4ST4EIBJZYVSWQ, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.961825236 (3h40m3.001398356s) [noise=true , address=45.91.116.7:4009 , piece_id=X2TGDVQGKFFJVM5UPKQ5TL7O76VQ7F3YTK3ZXD7F6NWCIXFRELCA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.961871174 (3h40m3.001444294s) [noise=true , address=45.91.116.13:4155 , piece_id=KVSLFHZ527LMCC4I2RB7SABZYMK4YQGA57K6D67VG254SOCTUMSQ, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.962043073 (3h40m3.001616193s) [noise=true , address=45.91.116.6:4975 , piece_id=MI5EIXGJSHALBJQCYJWJHQSDMHELUZJKUPNL4RUXZRZ6LWHKYSPA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.962093670 (3h40m3.0016668s) [noise=true , address=77.90.31.12:4592 , piece_id=E76LXWQJAMCYRZQ2B2ZTFNLZGNOOOOB7ADZMG4ZUC272WXUCAYNQ, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.962253967 (3h40m3.001827087s) [noise=true , address=45.91.116.7:4011 , piece_id=UTCBQCPJF7AVXKCCO5WFRWBKNI6PJZGOJS2JIAU5NQA3PPRFTDQQ, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.962541326 (3h40m3.002114446s) [noise=true , address=77.90.31.15:4697 , piece_id=XNWIQLO6WUMEDA6J6GZNMTDKJ5CAIMJI2EAJUDGXZ3JLYTCA5NEA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.962760135 (3h40m3.002333255s) [noise=true , address=45.91.116.8:4048 , piece_id=VU3QV7VQEWDFGRNLCYGYVFOOJG4RL4XQRZI7DASTUMZIH24DSQUA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:26.962949517 (3h40m3.002522637s) [noise=true , address=77.90.31.21:4879 , piece_id=ZO7ECYWFHRCD7XCZ6EOS6OH2NG3DTK2HHGIFQXUDQWKQZ5WY2CXA, seg_pos={0 0}, upload=3196]: Uploading piece... Sep 6 17:08:28.782128508 (3h40m4.821701628s) [upload=3196]: Next returned nil segment Sep 6 17:08:28.782146862 (3h40m4.821719982s) [upload=3196]: Waiting for error group managing segments... Sep 6 17:08:29.086638188 (3h40m5.126211308s) [noise=true , address=77.90.31.10:4520 , piece_id=X44SZGRTPHFHX24FDFBVBKRMD4RCZP4A6PN7SAYLQT5GIV4MRLQA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.118591722 (3h40m5.158164842s) [noise=true , address=45.91.116.13:4155 , piece_id=KVSLFHZ527LMCC4I2RB7SABZYMK4YQGA57K6D67VG254SOCTUMSQ, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.131657513 (3h40m5.171230633s) [noise=true , address=45.91.116.5:4299 , piece_id=SFQ4OFVFFEOCCWUEPYHSJT227DOYNG43NI2FHEKYLM2PTWLJWFSA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.141747511 (3h40m5.181320661s) [noise=true , address=45.91.116.7:4009 , piece_id=X2TGDVQGKFFJVM5UPKQ5TL7O76VQ7F3YTK3ZXD7F6NWCIXFRELCA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.180750593 (3h40m5.220323713s) [noise=true , address=45.91.116.6:4975 , piece_id=MI5EIXGJSHALBJQCYJWJHQSDMHELUZJKUPNL4RUXZRZ6LWHKYSPA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.207144909 (3h40m5.246718019s) [noise=true , address=45.91.116.11:4119 , piece_id=4QAHNDRRU2CYUQFVWUZ6QX6XDPXISDWOS2HPSGXY44MGZHA42T4A, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.216984887 (3h40m5.256558007s) [noise=true , address=45.91.116.8:4048 , piece_id=VU3QV7VQEWDFGRNLCYGYVFOOJG4RL4XQRZI7DASTUMZIH24DSQUA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.322551711 (3h40m5.362124821s) [noise=true , address=45.91.116.9:4261 , piece_id=6G2NRF52FE2K3PALYEC3CANKRHO7NDTPHV733KTI5O7SQDDZXNWA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.432594197 (3h40m5.472167317s) [noise=true , address=45.91.116.4:4937 , piece_id=NUHGSVRL4NY23PDRLNRIDR7BBDJNY6X2VML2LBKNMRVNKFMMRMNA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.438842063 (3h40m5.478415183s) [noise=true , address=45.91.116.7:4011 , piece_id=UTCBQCPJF7AVXKCCO5WFRWBKNI6PJZGOJS2JIAU5NQA3PPRFTDQQ, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.476747214 (3h40m5.516320334s) [noise=true , address=77.90.31.12:4592 , piece_id=E76LXWQJAMCYRZQ2B2ZTFNLZGNOOOOB7ADZMG4ZUC272WXUCAYNQ, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.531325830 (3h40m5.57089895s) [noise=true , address=77.90.31.21:4879 , piece_id=ZO7ECYWFHRCD7XCZ6EOS6OH2NG3DTK2HHGIFQXUDQWKQZ5WY2CXA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.619125606 (3h40m5.658698726s) [noise=true , address=77.90.31.10:4517 , piece_id=YCYEXOGV2V4JBRJO7UCKJP5ALBJHXWBGHR7Z6KO6CFJQJMH4ZMRA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: <nil> Sep 6 17:08:29.619153470 (3h40m5.65872659s) [seg_pos={0 0}, upload=3196]: Segment reached optimal threshold of 13 pieces. Sep 6 17:08:29.619271535 (3h40m5.658844645s) [noise=true , address=77.90.31.15:4697 , piece_id=XNWIQLO6WUMEDA6J6GZNMTDKJ5CAIMJI2EAJUDGXZ3JLYTCA5NEA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5): protocol: expected piece hash; context canceled; EOF Sep 6 17:08:29.619458623 (3h40m5.659031743s) [noise=true , address=77.90.31.16:4733 , piece_id=BSSUJWNUOAUUVJ6PBDUAY2VQPE3YAMQXXSBFWLWPI6QUNJAQXQWA, seg_pos={0 0}, upload=3196]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:51836->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:51836->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51836->77.90.31.16:4733: use of closed network connection Sep 6 17:08:29.619495864 (3h40m5.659068974s) [noise=true , address=45.91.116.10:4083 , piece_id=XO6E7AKAP2LVD4A4JOCP5TCBPZVATB4RMJXEMO4ST4EIBJZYVSWQ, seg_pos={0 0}, upload=3196]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:57014->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:57014->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57014->45.91.116.10:4083: use of closed network connection Sep 6 17:08:29.619506666 (3h40m5.659079786s) [seg_pos={0 0}, upload=3196]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 17:08:29.619565368 (3h40m5.659138488s) [upload=3196]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 17:08:29.619569155 (3h40m5.659142275s) [upload=3196]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 17:08:29.981128051 (3h40m6.020701171s) [upload=3196]: Done uploading segments... Sep 6 17:08:29.981146546 (3h40m6.020719666s) [upload=3196]: Upload finished. err: <nil> Duration: 30.002097663s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 24 out of 60 is failed (threshold=3) [XX..XX..X.X.XX....XX.XXX.....X..X....X......X.XX.X....XX..XX] duration: 30.00217383s FAIL (2025-09-06 17:08:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 17:07:56.399308104 (3h39m32.438881274s) [upload=3190]: Starting upload Sep 6 17:07:56.399440537 (3h39m32.439013657s) [upload=3190]: Done starting upload Sep 6 17:07:56.399504049 (3h39m32.439077169s) [upload=3190]: Uploading segments... Sep 6 17:07:56.399640440 (3h39m32.43921356s) [upload=3190]: Got next segment. Inline: false Sep 6 17:07:56.399672040 (3h39m32.43924515s) [upload=3190]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 17:07:56.399676358 (3h39m32.439249469s) [upload=3190]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 17:07:56.669029207 (3h39m32.708602327s) [seg_pos={0 0}, upload=3190]: Begin upload segment... Sep 6 17:07:56.669073643 (3h39m32.708646763s) [seg_pos={0 0}, upload=3190]: Done begin upload segment. Sep 6 17:07:56.669098150 (3h39m32.70867127s) [noise=true , address=45.91.116.4:4938 , piece_id=EQYIOUPK4MUOXL7TYHRXAHLBBATXSXTEQUJLLAOSJMILQ6WZMM3Q, seg_pos={0 0}, upload=3190]: Uploading piece... Sep 6 17:07:56.669129810 (3h39m32.70870293s) [noise=true , address=77.90.31.15:4697 , piece_id=E2UMCWCOLFDO5LJ7DQZCDYQZ2LNCGFZ2U3ACGA3CBFP26ZQZG6DQ, seg_pos={0 0}, upload=3190]: Uploading piece... Sep 6 17:07:56.669468848 (3h39m32.709041968s) [noise=true , address=45.91.116.11:4119 , piece_id=XZ64BDAWIB2AUIYZKPWKAXYI7VKNPVHR5WFAXQCFWH4XNDLRPB5Q, seg_pos={0 0}, upload=3190]: Uploading piece... Sep 6 17:07:56.669571074 (3h39m32.709144194s) [noise=true , address=45.91.116.5:4299 , piece_id=5TTEKZBZ4CNNQZ75I2YDV2HW762PZEDWPAVNV6HHGTH63UYZZNHA, seg_pos={0 0}, upload=3190]: Uploading piece... Sep 6 17:07:56.669736511 (3h39m32.709309621s) [noise=true , address=45.91.116.9:4263 , piece_id=SP6QPETADNVI4USLDGKB6RFOHXM4GWWMWX44ULWWJETYQPTHZM7A, seg_pos={0 0}, upload=3190]: Uploading piece... Sep 6 17:07:56.669860558 (3h39m32.709433678s) [noise=true , address=45.91.116.8:4048 , piece_id=UABQA2ZSEEFYRY3CPYKVIC572SQU366KW7SKX4GZMO2BNJ7TL3XQ, seg_pos={0 0}, upload=3190]: Uploading piece... Sep 6 17:07:56.670086070 (3h39m32.70965919s) [noise=true , address=45.91.116.4:4940 , piece_id=JKBUO3CDTAQ63C7UTNHV3JVCJMYBQENHPTV33LNP2WRXHRIB2HYA, seg_pos={0 0}, upload=3190]: Uploading piece... Sep 6 17:07:56.670134091 (3h39m32.709707211s) [noise=true , address=45.91.116.5:4297 , piece_id=3GUO5IN5ZSWPEN6UH7LEZC3FN3IJK6TZWT7LLNOUIRFMPT5NBPQQ, seg_pos={0 0}, upload=3190]: Uploading piece... Sep 6 17:07:56.670425930 (3h39m32.70999905s) [noise=true , address=77.90.31.16:4733 , piece_id=AOXKZGJZNH7OFP2S6AFFFRY5KATDUCEBHWWBUGUKQEKQMZ7AALLQ, seg_pos={0 0}, upload=3190]: Uploading piece... Sep 6 17:07:56.670657463 (3h39m32.710230583s) [noise=true , address=45.91.116.7:4010 , piece_id=7WGKCKLZ3B4OVY6WKP7PEKOS5KP4PPIIORUV6RUC4XYGR4KLAGJA, seg_pos={0 0}, upload=3190]: Uploading piece... Sep 6 17:07:56.670938541 (3h39m32.710511651s) [noise=true , address=45.91.116.9:4262 , piece_id=F4CF2ORGCUJO2I7IFY23S6G2UYOTDROE3RRKYED2IW3WYTPZDVWA, seg_pos={0 0}, upload=3190]: Uploading piece... Sep 6 17:07:56.671152971 (3h39m32.710726091s) [noise=true , address=45.91.116.14:4191 , piece_id=S5CQ3MQFUETV2BCKYH5BBIXYFAKECZB2THG6LJSCTN6BU7HUUB5A, seg_pos={0 0}, upload=3190]: Uploading piece... Sep 6 17:07:56.671370868 (3h39m32.710943978s) [noise=true , address=77.90.31.12:4589 , piece_id=PRDU2OZR7YRJZG43UAODJLAEETBXWHUMIEHKEU4L2OAWCGFEWUTA, seg_pos={0 0}, upload=3190]: Uploading piece... Sep 6 17:07:56.671572884 (3h39m32.711146005s) [noise=true , address=45.91.116.14:4190 , piece_id=IOLPQLWCJVUIYNOEOW6NUG3IMZZB7AGEJBVMFGL74DTGSZDOCZUA, seg_pos={0 0}, upload=3190]: Uploading piece... Sep 6 17:07:56.671760313 (3h39m32.711333433s) [noise=true , address=45.91.116.10:4082 , piece_id=TOVPMY3IGPK5K2O3IDMWZIR3LCCLY3QA5UIGAWHL7ILQHY6SJSNQ, seg_pos={0 0}, upload=3190]: Uploading piece... Sep 6 17:07:56.672034137 (3h39m32.711607257s) [noise=true , address=45.91.116.13:4153 , piece_id=ZLILN4NGYPLB7OAYJZVQJI6M5EOKSX43GVV3JWLQFVNN5LFACV3Q, seg_pos={0 0}, upload=3190]: Uploading piece... Sep 6 17:07:57.747740938 (3h39m33.787314058s) [noise=true , address=77.90.31.12:4589 , piece_id=PRDU2OZR7YRJZG43UAODJLAEETBXWHUMIEHKEU4L2OAWCGFEWUTA, seg_pos={0 0}, upload=3190]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: storage node overloaded, request limit: 2000 Sep 6 17:07:57.944664092 (3h39m33.984237212s) [noise=true , address=77.90.31.16:4733 , piece_id=AOXKZGJZNH7OFP2S6AFFFRY5KATDUCEBHWWBUGUKQEKQMZ7AALLQ, seg_pos={0 0}, upload=3190]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:56844->77.90.31.16:4733: write: connection refused; manager closed: conn closed: op Sep 6 17:07:58.509606121 (3h39m34.549179241s) [upload=3190]: Next returned nil segment Sep 6 17:07:58.509626721 (3h39m34.549199841s) [upload=3190]: Waiting for error group managing segments... Sep 6 17:07:58.809609521 (3h39m34.849182641s) [noise=true , address=45.91.116.14:4190 , piece_id=IOLPQLWCJVUIYNOEOW6NUG3IMZZB7AGEJBVMFGL74DTGSZDOCZUA, seg_pos={0 0}, upload=3190]: Done uploading piece. err: <nil> Sep 6 17:07:58.830029393 (3h39m34.869602513s) [noise=true , address=45.91.116.5:4297 , piece_id=3GUO5IN5ZSWPEN6UH7LEZC3FN3IJK6TZWT7LLNOUIRFMPT5NBPQQ, seg_pos={0 0}, upload=3190]: Done uploading piece. err: <nil> Sep 6 17:07:58.918630232 (3h39m34.958203352s) [noise=true , address=45.91.116.5:4299 , piece_id=5TTEKZBZ4CNNQZ75I2YDV2HW762PZEDWPAVNV6HHGTH63UYZZNHA, seg_pos={0 0}, upload=3190]: Done uploading piece. err: <nil> Sep 6 17:07:58.960331988 (3h39m34.999905108s) [noise=true , address=45.91.116.13:4153 , piece_id=ZLILN4NGYPLB7OAYJZVQJI6M5EOKSX43GVV3JWLQFVNN5LFACV3Q, seg_pos={0 0}, upload=3190]: Done uploading piece. err: <nil> Sep 6 17:07:58.997527361 (3h39m35.037100481s) [noise=true , address=45.91.116.7:4010 , piece_id=7WGKCKLZ3B4OVY6WKP7PEKOS5KP4PPIIORUV6RUC4XYGR4KLAGJA, seg_pos={0 0}, upload=3190]: Done uploading piece. err: <nil> Sep 6 17:07:59.022948886 (3h39m35.062521996s) [noise=true , address=45.91.116.9:4263 , piece_id=SP6QPETADNVI4USLDGKB6RFOHXM4GWWMWX44ULWWJETYQPTHZM7A, seg_pos={0 0}, upload=3190]: Done uploading piece. err: <nil> Sep 6 17:07:59.040123280 (3h39m35.0796964s) [noise=true , address=45.91.116.11:4119 , piece_id=XZ64BDAWIB2AUIYZKPWKAXYI7VKNPVHR5WFAXQCFWH4XNDLRPB5Q, seg_pos={0 0}, upload=3190]: Done uploading piece. err: <nil> Sep 6 17:07:59.064417446 (3h39m35.103990566s) [noise=true , address=45.91.116.4:4940 , piece_id=JKBUO3CDTAQ63C7UTNHV3JVCJMYBQENHPTV33LNP2WRXHRIB2HYA, seg_pos={0 0}, upload=3190]: Done uploading piece. err: <nil> Sep 6 17:07:59.083373429 (3h39m35.122946549s) [noise=true , address=45.91.116.10:4082 , piece_id=TOVPMY3IGPK5K2O3IDMWZIR3LCCLY3QA5UIGAWHL7ILQHY6SJSNQ, seg_pos={0 0}, upload=3190]: Done uploading piece. err: <nil> Sep 6 17:07:59.088454892 (3h39m35.128028002s) [noise=true , address=77.90.31.15:4697 , piece_id=E2UMCWCOLFDO5LJ7DQZCDYQZ2LNCGFZ2U3ACGA3CBFP26ZQZG6DQ, seg_pos={0 0}, upload=3190]: Done uploading piece. err: <nil> Sep 6 17:07:59.101132752 (3h39m35.140705862s) [noise=true , address=45.91.116.9:4262 , piece_id=F4CF2ORGCUJO2I7IFY23S6G2UYOTDROE3RRKYED2IW3WYTPZDVWA, seg_pos={0 0}, upload=3190]: Done uploading piece. err: <nil> Sep 6 17:07:59.271192521 (3h39m35.310765631s) [noise=true , address=45.91.116.4:4938 , piece_id=EQYIOUPK4MUOXL7TYHRXAHLBBATXSXTEQUJLLAOSJMILQ6WZMM3Q, seg_pos={0 0}, upload=3190]: Done uploading piece. err: <nil> Sep 6 17:07:59.284712572 (3h39m35.324285692s) [noise=true , address=45.91.116.8:4048 , piece_id=UABQA2ZSEEFYRY3CPYKVIC572SQU366KW7SKX4GZMO2BNJ7TL3XQ, seg_pos={0 0}, upload=3190]: Done uploading piece. err: <nil> Sep 6 17:07:59.284732170 (3h39m35.32430529s) [seg_pos={0 0}, upload=3190]: Segment reached optimal threshold of 13 pieces. Sep 6 17:07:59.285145781 (3h39m35.324718901s) [noise=true , address=45.91.116.14:4191 , piece_id=S5CQ3MQFUETV2BCKYH5BBIXYFAKECZB2THG6LJSCTN6BU7HUUB5A, seg_pos={0 0}, upload=3190]: Done uploading piece. err: <nil> Sep 6 17:07:59.285197921 (3h39m35.324771051s) [seg_pos={0 0}, upload=3190]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 6 17:07:59.285325926 (3h39m35.324899046s) [upload=3190]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 17:07:59.285331958 (3h39m35.324905088s) [upload=3190]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 17:07:59.683906351 (3h39m35.723479471s) [upload=3190]: Done uploading segments... Sep 6 17:07:59.683926800 (3h39m35.72349992s) [upload=3190]: Upload finished. err: <nil> Duration: 30.002186575s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [.XX..XX..X.X.XX....XX.XXX.....X..X....X......X.XX.X....XX..X] duration: 30.001692939s FAIL (2025-09-06 17:07:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 17:06:50.148381207 (3h38m26.187954327s) [upload=3176]: Starting upload Sep 6 17:06:50.148439549 (3h38m26.188012659s) [upload=3176]: Done starting upload Sep 6 17:06:50.148542135 (3h38m26.188115245s) [upload=3176]: Uploading segments... Sep 6 17:06:50.148624593 (3h38m26.188197713s) [upload=3176]: Got next segment. Inline: false Sep 6 17:06:50.148658367 (3h38m26.188231488s) [upload=3176]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 17:06:50.148663026 (3h38m26.188236146s) [upload=3176]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 17:06:51.445176963 (3h38m27.484750083s) [seg_pos={0 0}, upload=3176]: Begin upload segment... Sep 6 17:06:51.445218904 (3h38m27.484792024s) [seg_pos={0 0}, upload=3176]: Done begin upload segment. Sep 6 17:06:51.445241467 (3h38m27.484814587s) [noise=true , address=45.91.116.12:4225 , piece_id=O6AQEL26VJC5R3V26TJVKDS3V2NKEPHQ4ALXF4MP5QMXDMTR6L7Q, seg_pos={0 0}, upload=3176]: Uploading piece... Sep 6 17:06:51.445631644 (3h38m27.485204754s) [noise=true , address=45.91.116.6:4975 , piece_id=JHGFKHVJEPLMZ37LCIAGKQ5RW6WR2ATPFRWVWVMTCRPTPUL2B4PQ, seg_pos={0 0}, upload=3176]: Uploading piece... Sep 6 17:06:51.445937469 (3h38m27.485510589s) [noise=true , address=77.90.31.15:4698 , piece_id=OBK2BDMXXWVRMFRNQB6RDJE7GEGYNMCIPAQMANZDI7H67M3RCEUA, seg_pos={0 0}, upload=3176]: Uploading piece... Sep 6 17:06:51.446244306 (3h38m27.485817426s) [noise=true , address=77.90.31.21:4878 , piece_id=RWVAIJHXVANCN65HXCCL7T3WS24MJUBLCG6O4SEAMWBCVCL46Q7Q, seg_pos={0 0}, upload=3176]: Uploading piece... Sep 6 17:06:51.446765763 (3h38m27.486338883s) [noise=true , address=45.91.116.6:4974 , piece_id=3IS5FNR3DAR6FCFPEXKZ7XJHG5HWADQ7ECQSLUPY32E5CAM7LQBQ, seg_pos={0 0}, upload=3176]: Uploading piece... Sep 6 17:06:51.447309634 (3h38m27.486882754s) [noise=true , address=77.90.31.21:4879 , piece_id=RSV3USI7X4TWN64HGNM7WSKOXWW7E72N5M2JNOK7AIXXBP4QMLJQ, seg_pos={0 0}, upload=3176]: Uploading piece... Sep 6 17:06:51.447587185 (3h38m27.487160305s) [noise=true , address=77.90.31.14:4662 , piece_id=YJRLUF7NHKDEPXPPHMEZZQIDV7GZA4A33VU4ZJCMT6IW5FBIJNNQ, seg_pos={0 0}, upload=3176]: Uploading piece... Sep 6 17:06:51.447809622 (3h38m27.487382742s) [noise=true , address=45.91.116.14:4190 , piece_id=H3YYY27G63ACY54OETMU7Q5FTOST63LYO6JDMGIQPDKIG5EXA4PQ, seg_pos={0 0}, upload=3176]: Uploading piece... Sep 6 17:06:51.448212422 (3h38m27.487785542s) [noise=true , address=77.90.31.16:4734 , piece_id=7SYE6275JEC3CH6FPCKWMMCJBBFYXW2IJET3IACBME7UHBPUPDSA, seg_pos={0 0}, upload=3176]: Uploading piece... Sep 6 17:06:51.448516794 (3h38m27.488089914s) [noise=true , address=45.91.116.13:4155 , piece_id=JFMPGGJOZUWSGAYLIGGR2AZN6XDFGQGGXFWLBFITVZLUPLJHQ2EQ, seg_pos={0 0}, upload=3176]: Uploading piece... Sep 6 17:06:51.448843529 (3h38m27.488416649s) [noise=true , address=45.91.116.8:4047 , piece_id=V6DRSPINIDSYV6BRP25AOQGFD5K74XHOAPNXXRSW77M3Z76WV7MQ, seg_pos={0 0}, upload=3176]: Uploading piece... Sep 6 17:06:51.449071085 (3h38m27.488644205s) [noise=true , address=45.91.116.9:4262 , piece_id=COQ4E27TXMXKGZ2N5HQON45ERB6J3F2HH3PYBUUI62NE6NK4LXEA, seg_pos={0 0}, upload=3176]: Uploading piece... Sep 6 17:06:51.449303229 (3h38m27.488876349s) [noise=true , address=45.91.116.4:4940 , piece_id=UT7EJ4E2ARC5MT2DNHGUPY5LNLZCDLYETDQG3JH2IJ3LRJXXLJEA, seg_pos={0 0}, upload=3176]: Uploading piece... Sep 6 17:06:51.449556614 (3h38m27.489129734s) [noise=true , address=45.91.116.5:4298 , piece_id=PJVEV2VC73SJ6FFKAL5S7ZWUKT62EPNMAJGIHUKGGRDBQZTGTZLA, seg_pos={0 0}, upload=3176]: Uploading piece... Sep 6 17:06:51.450052614 (3h38m27.489625734s) [noise=true , address=45.91.116.14:4191 , piece_id=7BWV33ADJ7PBXRLKEF6UUV3CKBSI4SCC6RUQFWYMUM2J5A2ITKNQ, seg_pos={0 0}, upload=3176]: Uploading piece... Sep 6 17:06:51.450418654 (3h38m27.489991774s) [noise=true , address=45.91.116.11:4119 , piece_id=PWXCNCS5SH7HPQIFSCXFAXYDU3QSP6GQOS3QLT7RIS7ALI5YHQZQ, seg_pos={0 0}, upload=3176]: Uploading piece... Sep 6 17:06:53.270589031 (3h38m29.310162171s) [upload=3176]: Next returned nil segment Sep 6 17:06:53.270621032 (3h38m29.310194162s) [upload=3176]: Waiting for error group managing segments... Sep 6 17:06:53.577743684 (3h38m29.617316804s) [noise=true , address=77.90.31.14:4662 , piece_id=YJRLUF7NHKDEPXPPHMEZZQIDV7GZA4A33VU4ZJCMT6IW5FBIJNNQ, seg_pos={0 0}, upload=3176]: Done uploading piece. err: <nil> Sep 6 17:06:53.620975959 (3h38m29.660549069s) [noise=true , address=45.91.116.14:4190 , piece_id=H3YYY27G63ACY54OETMU7Q5FTOST63LYO6JDMGIQPDKIG5EXA4PQ, seg_pos={0 0}, upload=3176]: Done uploading piece. err: <nil> Sep 6 17:06:53.655076741 (3h38m29.694649861s) [noise=true , address=45.91.116.6:4975 , piece_id=JHGFKHVJEPLMZ37LCIAGKQ5RW6WR2ATPFRWVWVMTCRPTPUL2B4PQ, seg_pos={0 0}, upload=3176]: Done uploading piece. err: <nil> Sep 6 17:06:53.691541887 (3h38m29.731115007s) [noise=true , address=45.91.116.14:4191 , piece_id=7BWV33ADJ7PBXRLKEF6UUV3CKBSI4SCC6RUQFWYMUM2J5A2ITKNQ, seg_pos={0 0}, upload=3176]: Done uploading piece. err: <nil> Sep 6 17:06:53.711927114 (3h38m29.751500234s) [noise=true , address=45.91.116.5:4298 , piece_id=PJVEV2VC73SJ6FFKAL5S7ZWUKT62EPNMAJGIHUKGGRDBQZTGTZLA, seg_pos={0 0}, upload=3176]: Done uploading piece. err: <nil> Sep 6 17:06:53.733902113 (3h38m29.773475233s) [noise=true , address=45.91.116.9:4262 , piece_id=COQ4E27TXMXKGZ2N5HQON45ERB6J3F2HH3PYBUUI62NE6NK4LXEA, seg_pos={0 0}, upload=3176]: Done uploading piece. err: <nil> Sep 6 17:06:53.750497268 (3h38m29.790070408s) [noise=true , address=45.91.116.13:4155 , piece_id=JFMPGGJOZUWSGAYLIGGR2AZN6XDFGQGGXFWLBFITVZLUPLJHQ2EQ, seg_pos={0 0}, upload=3176]: Done uploading piece. err: <nil> Sep 6 17:06:53.767424589 (3h38m29.806997709s) [noise=true , address=45.91.116.12:4225 , piece_id=O6AQEL26VJC5R3V26TJVKDS3V2NKEPHQ4ALXF4MP5QMXDMTR6L7Q, seg_pos={0 0}, upload=3176]: Done uploading piece. err: <nil> Sep 6 17:06:53.774212628 (3h38m29.813785748s) [noise=true , address=45.91.116.11:4119 , piece_id=PWXCNCS5SH7HPQIFSCXFAXYDU3QSP6GQOS3QLT7RIS7ALI5YHQZQ, seg_pos={0 0}, upload=3176]: Done uploading piece. err: <nil> Sep 6 17:06:53.787360696 (3h38m29.826933817s) [noise=true , address=45.91.116.6:4974 , piece_id=3IS5FNR3DAR6FCFPEXKZ7XJHG5HWADQ7ECQSLUPY32E5CAM7LQBQ, seg_pos={0 0}, upload=3176]: Done uploading piece. err: <nil> Sep 6 17:06:53.794755657 (3h38m29.834328777s) [noise=true , address=77.90.31.21:4879 , piece_id=RSV3USI7X4TWN64HGNM7WSKOXWW7E72N5M2JNOK7AIXXBP4QMLJQ, seg_pos={0 0}, upload=3176]: Done uploading piece. err: <nil> Sep 6 17:06:53.836335541 (3h38m29.875908661s) [noise=true , address=77.90.31.21:4878 , piece_id=RWVAIJHXVANCN65HXCCL7T3WS24MJUBLCG6O4SEAMWBCVCL46Q7Q, seg_pos={0 0}, upload=3176]: Done uploading piece. err: <nil> Sep 6 17:06:53.969897262 (3h38m30.009470382s) [noise=true , address=45.91.116.8:4047 , piece_id=V6DRSPINIDSYV6BRP25AOQGFD5K74XHOAPNXXRSW77M3Z76WV7MQ, seg_pos={0 0}, upload=3176]: Done uploading piece. err: <nil> Sep 6 17:06:53.969924122 (3h38m30.009497242s) [seg_pos={0 0}, upload=3176]: Segment reached optimal threshold of 13 pieces. Sep 6 17:06:53.970040566 (3h38m30.009613686s) [noise=true , address=77.90.31.15:4698 , piece_id=OBK2BDMXXWVRMFRNQB6RDJE7GEGYNMCIPAQMANZDI7H67M3RCEUA, seg_pos={0 0}, upload=3176]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Sep 6 17:06:53.970146669 (3h38m30.009719779s) [noise=true , address=77.90.31.16:4734 , piece_id=7SYE6275JEC3CH6FPCKWMMCJBBFYXW2IJET3IACBME7UHBPUPDSA, seg_pos={0 0}, upload=3176]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: context canceled; context canceled Sep 6 17:06:53.970250267 (3h38m30.009823377s) [noise=true , address=45.91.116.4:4940 , piece_id=UT7EJ4E2ARC5MT2DNHGUPY5LNLZCDLYETDQG3JH2IJ3LRJXXLJEA, seg_pos={0 0}, upload=3176]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe): protocol: write tcp 188.166.225.170:55848->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:55848->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55848->45.91.116.4:4940: use of closed network connection Sep 6 17:06:53.970291736 (3h38m30.009864856s) [seg_pos={0 0}, upload=3176]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 17:06:53.970367151 (3h38m30.009940271s) [upload=3176]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 17:06:53.970371810 (3h38m30.00994492s) [upload=3176]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 17:06:54.484426105 (3h38m30.523999225s) [upload=3176]: Done uploading segments... Sep 6 17:06:54.484445833 (3h38m30.524018953s) [upload=3176]: Upload finished. err: <nil> Duration: 30.00171975s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [..XX..XX..X.X.XX....XX.XXX.....X..X....X......X.XX.X....XX..] duration: 13.392986002s FAIL (2025-09-06 17:06:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 17:04:48.767994646 (3h36m24.807567776s) [upload=3145]: Starting upload Sep 6 17:04:48.768093364 (3h36m24.807666495s) [upload=3145]: Done starting upload Sep 6 17:04:48.768203756 (3h36m24.807776876s) [upload=3145]: Uploading segments... Sep 6 17:04:48.768374243 (3h36m24.807947363s) [upload=3145]: Got next segment. Inline: false Sep 6 17:04:48.768409871 (3h36m24.807982991s) [upload=3145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 17:04:48.768414609 (3h36m24.80798773s) [upload=3145]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 17:04:49.042040494 (3h36m25.081613604s) [seg_pos={0 0}, upload=3145]: Begin upload segment... Sep 6 17:04:49.042081854 (3h36m25.081654974s) [seg_pos={0 0}, upload=3145]: Done begin upload segment. Sep 6 17:04:49.042103194 (3h36m25.081676314s) [noise=true , address=77.90.31.17:4770 , piece_id=QVFTS75FYPHJTPMBWU47FSBQJOXP6NPB7NP7YHWFTL7C2JKRRPWA, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.042130767 (3h36m25.081703877s) [noise=true , address=77.90.31.10:4518 , piece_id=EAPBMGI4EX4GHIHSCNZLPCAOLVZOXH2Y6RPJ465ARFBEFJWO2CGQ, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.042462482 (3h36m25.082035602s) [noise=true , address=77.90.31.21:4877 , piece_id=AG5DFWZQDLZDSJN2TAE7N5KI7BGTR2OXWPXQF46ETT55UWKNVRWA, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.042888778 (3h36m25.082461898s) [noise=true , address=45.91.116.7:4009 , piece_id=3N55SSNQRUSY3LEXOYL7RIW22LDZJIFASU6EEWQ4TNZUOCBMHJ7Q, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.043698517 (3h36m25.083271657s) [noise=true , address=45.91.116.5:4297 , piece_id=4ZBL7Y5LQUIR2UJDZ3F6LBE7P4VXFUUH4GMNGFZRKU2BRSERLUSA, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.044328853 (3h36m25.083902004s) [noise=true , address=77.90.31.14:4662 , piece_id=Y4WBWHEFMJY45L7TU7O4DAANFNFL4TEHYZUR2M77IN7WFVV4SA5Q, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.044920426 (3h36m25.084493576s) [noise=true , address=45.91.116.12:4226 , piece_id=3DS4CCYXPKYXIBLR6SH6ON26KHTIVBRR3MNVXBEOD4X2IH2TKM4A, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.045526355 (3h36m25.085099495s) [noise=true , address=45.91.116.11:4117 , piece_id=WNPSPOJZU7E3HB6CKM37FKG5AGLKAWOG5ZHVF5PL6O3JV7TKVIXQ, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.046803047 (3h36m25.086376167s) [noise=true , address=45.91.116.8:4045 , piece_id=XZFGOL6I2FCZIR3KE7ELGE6AONVMQQZ4MFUXVQEE63MMMG76NSZA, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.047215998 (3h36m25.086789118s) [noise=true , address=45.91.116.5:4299 , piece_id=JI7R6IU7YNTTHORLJZWLILS7N4RBGXVMOSQFHSUBWJCMTC42VJEQ, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.047482037 (3h36m25.087055157s) [noise=true , address=45.91.116.11:4119 , piece_id=N4TGQKPTAXXVFOXAMQOS342SKPFHGTD2A5JW47XX73P3DGJHUXGA, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.047778905 (3h36m25.087352025s) [noise=true , address=77.90.31.15:4699 , piece_id=G752LE6G55SFWLGWGWMOBV3OCJQYLIDLTUPAQSWPTGIKZVIHCS3Q, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.048024465 (3h36m25.087597785s) [noise=true , address=45.91.116.14:4191 , piece_id=BCPIMZPE4YU63QMM5OT3BX7EENQMXNBWXPLZEGD3ICMKUFHH7S7A, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.048291867 (3h36m25.087864987s) [noise=true , address=45.91.116.11:4118 , piece_id=KOI23FVX43DPEK6Q653EGNV2WDHNQ3MFW6L75H2EYJ4WFCH5WVFA, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.048544390 (3h36m25.08811752s) [noise=true , address=77.90.31.12:4589 , piece_id=HH5M2XL2536CRNV7AX2QGOABD2F36QQQB3FG3ZQUBHEN527WAEJA, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.048791262 (3h36m25.088364382s) [noise=true , address=45.91.116.9:4261 , piece_id=2GEQPO5XWN57B5UAVC5WTCMIHCIYZDW4CSGMQO4EXTHK4A2RGTLQ, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:50.840862672 (3h36m26.880435792s) [upload=3145]: Next returned nil segment Sep 6 17:04:50.840881569 (3h36m26.880454689s) [upload=3145]: Waiting for error group managing segments... Sep 6 17:04:51.216997468 (3h36m27.256570588s) [noise=true , address=45.91.116.9:4261 , piece_id=2GEQPO5XWN57B5UAVC5WTCMIHCIYZDW4CSGMQO4EXTHK4A2RGTLQ, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.241113665 (3h36m27.280686785s) [noise=true , address=45.91.116.14:4191 , piece_id=BCPIMZPE4YU63QMM5OT3BX7EENQMXNBWXPLZEGD3ICMKUFHH7S7A, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.243256716 (3h36m27.282829836s) [noise=true , address=77.90.31.15:4699 , piece_id=G752LE6G55SFWLGWGWMOBV3OCJQYLIDLTUPAQSWPTGIKZVIHCS3Q, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.263575827 (3h36m27.303148937s) [noise=true , address=77.90.31.12:4589 , piece_id=HH5M2XL2536CRNV7AX2QGOABD2F36QQQB3FG3ZQUBHEN527WAEJA, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.270086886 (3h36m27.309660006s) [noise=true , address=45.91.116.5:4299 , piece_id=JI7R6IU7YNTTHORLJZWLILS7N4RBGXVMOSQFHSUBWJCMTC42VJEQ, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.277565416 (3h36m27.317138536s) [noise=true , address=45.91.116.8:4045 , piece_id=XZFGOL6I2FCZIR3KE7ELGE6AONVMQQZ4MFUXVQEE63MMMG76NSZA, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.293258204 (3h36m27.332831335s) [noise=true , address=45.91.116.11:4118 , piece_id=KOI23FVX43DPEK6Q653EGNV2WDHNQ3MFW6L75H2EYJ4WFCH5WVFA, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.312504762 (3h36m27.352077882s) [noise=true , address=77.90.31.10:4518 , piece_id=EAPBMGI4EX4GHIHSCNZLPCAOLVZOXH2Y6RPJ465ARFBEFJWO2CGQ, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.333552437 (3h36m27.373125557s) [noise=true , address=77.90.31.14:4662 , piece_id=Y4WBWHEFMJY45L7TU7O4DAANFNFL4TEHYZUR2M77IN7WFVV4SA5Q, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.334919302 (3h36m27.374492423s) [noise=true , address=45.91.116.11:4117 , piece_id=WNPSPOJZU7E3HB6CKM37FKG5AGLKAWOG5ZHVF5PL6O3JV7TKVIXQ, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.377061653 (3h36m27.416634773s) [noise=true , address=45.91.116.7:4009 , piece_id=3N55SSNQRUSY3LEXOYL7RIW22LDZJIFASU6EEWQ4TNZUOCBMHJ7Q, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.395711139 (3h36m27.435284259s) [noise=true , address=45.91.116.5:4297 , piece_id=4ZBL7Y5LQUIR2UJDZ3F6LBE7P4VXFUUH4GMNGFZRKU2BRSERLUSA, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.417020304 (3h36m27.456593424s) [noise=true , address=45.91.116.12:4226 , piece_id=3DS4CCYXPKYXIBLR6SH6ON26KHTIVBRR3MNVXBEOD4X2IH2TKM4A, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.417036926 (3h36m27.456610046s) [seg_pos={0 0}, upload=3145]: Segment reached optimal threshold of 13 pieces. Sep 6 17:04:51.417139733 (3h36m27.456712853s) [noise=true , address=45.91.116.11:4119 , piece_id=N4TGQKPTAXXVFOXAMQOS342SKPFHGTD2A5JW47XX73P3DGJHUXGA, seg_pos={0 0}, upload=3145]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: expected piece hash; context canceled; EOF Sep 6 17:04:51.417288377 (3h36m27.456861497s) [noise=true , address=77.90.31.21:4877 , piece_id=AG5DFWZQDLZDSJN2TAE7N5KI7BGTR2OXWPXQF46ETT55UWKNVRWA, seg_pos={0 0}, upload=3145]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Sep 6 17:04:51.417368490 (3h36m27.45694161s) [noise=true , address=77.90.31.17:4770 , piece_id=QVFTS75FYPHJTPMBWU47FSBQJOXP6NPB7NP7YHWFTL7C2JKRRPWA, seg_pos={0 0}, upload=3145]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: write tcp 188.166.225.170:33928->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:33928->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33928->77.90.31.17:4770: use of closed network connection Sep 6 17:04:51.417380063 (3h36m27.456953183s) [seg_pos={0 0}, upload=3145]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 17:04:51.417442171 (3h36m27.457015281s) [upload=3145]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 17:04:51.417446028 (3h36m27.457019149s) [upload=3145]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 17:04:51.994441372 (3h36m28.034014502s) [upload=3145]: Done uploading segments... Sep 6 17:04:51.994466570 (3h36m28.03403974s) [upload=3145]: Upload finished. err: <nil> Duration: 13.393033253s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [...XX..XX..X.X.XX....XX.XXX.....X..X....X......X.XX.X....XX.] duration: 14.439662164s FAIL (2025-09-06 17:05:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 17:04:48.767994646 (3h36m24.807567776s) [upload=3145]: Starting upload Sep 6 17:04:48.768093364 (3h36m24.807666495s) [upload=3145]: Done starting upload Sep 6 17:04:48.768203756 (3h36m24.807776876s) [upload=3145]: Uploading segments... Sep 6 17:04:48.768374243 (3h36m24.807947363s) [upload=3145]: Got next segment. Inline: false Sep 6 17:04:48.768409871 (3h36m24.807982991s) [upload=3145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 17:04:48.768414609 (3h36m24.80798773s) [upload=3145]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 17:04:49.042040494 (3h36m25.081613604s) [seg_pos={0 0}, upload=3145]: Begin upload segment... Sep 6 17:04:49.042081854 (3h36m25.081654974s) [seg_pos={0 0}, upload=3145]: Done begin upload segment. Sep 6 17:04:49.042103194 (3h36m25.081676314s) [noise=true , address=77.90.31.17:4770 , piece_id=QVFTS75FYPHJTPMBWU47FSBQJOXP6NPB7NP7YHWFTL7C2JKRRPWA, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.042130767 (3h36m25.081703877s) [noise=true , address=77.90.31.10:4518 , piece_id=EAPBMGI4EX4GHIHSCNZLPCAOLVZOXH2Y6RPJ465ARFBEFJWO2CGQ, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.042462482 (3h36m25.082035602s) [noise=true , address=77.90.31.21:4877 , piece_id=AG5DFWZQDLZDSJN2TAE7N5KI7BGTR2OXWPXQF46ETT55UWKNVRWA, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.042888778 (3h36m25.082461898s) [noise=true , address=45.91.116.7:4009 , piece_id=3N55SSNQRUSY3LEXOYL7RIW22LDZJIFASU6EEWQ4TNZUOCBMHJ7Q, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.043698517 (3h36m25.083271657s) [noise=true , address=45.91.116.5:4297 , piece_id=4ZBL7Y5LQUIR2UJDZ3F6LBE7P4VXFUUH4GMNGFZRKU2BRSERLUSA, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.044328853 (3h36m25.083902004s) [noise=true , address=77.90.31.14:4662 , piece_id=Y4WBWHEFMJY45L7TU7O4DAANFNFL4TEHYZUR2M77IN7WFVV4SA5Q, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.044920426 (3h36m25.084493576s) [noise=true , address=45.91.116.12:4226 , piece_id=3DS4CCYXPKYXIBLR6SH6ON26KHTIVBRR3MNVXBEOD4X2IH2TKM4A, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.045526355 (3h36m25.085099495s) [noise=true , address=45.91.116.11:4117 , piece_id=WNPSPOJZU7E3HB6CKM37FKG5AGLKAWOG5ZHVF5PL6O3JV7TKVIXQ, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.046803047 (3h36m25.086376167s) [noise=true , address=45.91.116.8:4045 , piece_id=XZFGOL6I2FCZIR3KE7ELGE6AONVMQQZ4MFUXVQEE63MMMG76NSZA, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.047215998 (3h36m25.086789118s) [noise=true , address=45.91.116.5:4299 , piece_id=JI7R6IU7YNTTHORLJZWLILS7N4RBGXVMOSQFHSUBWJCMTC42VJEQ, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.047482037 (3h36m25.087055157s) [noise=true , address=45.91.116.11:4119 , piece_id=N4TGQKPTAXXVFOXAMQOS342SKPFHGTD2A5JW47XX73P3DGJHUXGA, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.047778905 (3h36m25.087352025s) [noise=true , address=77.90.31.15:4699 , piece_id=G752LE6G55SFWLGWGWMOBV3OCJQYLIDLTUPAQSWPTGIKZVIHCS3Q, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.048024465 (3h36m25.087597785s) [noise=true , address=45.91.116.14:4191 , piece_id=BCPIMZPE4YU63QMM5OT3BX7EENQMXNBWXPLZEGD3ICMKUFHH7S7A, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.048291867 (3h36m25.087864987s) [noise=true , address=45.91.116.11:4118 , piece_id=KOI23FVX43DPEK6Q653EGNV2WDHNQ3MFW6L75H2EYJ4WFCH5WVFA, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.048544390 (3h36m25.08811752s) [noise=true , address=77.90.31.12:4589 , piece_id=HH5M2XL2536CRNV7AX2QGOABD2F36QQQB3FG3ZQUBHEN527WAEJA, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.048791262 (3h36m25.088364382s) [noise=true , address=45.91.116.9:4261 , piece_id=2GEQPO5XWN57B5UAVC5WTCMIHCIYZDW4CSGMQO4EXTHK4A2RGTLQ, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:50.840862672 (3h36m26.880435792s) [upload=3145]: Next returned nil segment Sep 6 17:04:50.840881569 (3h36m26.880454689s) [upload=3145]: Waiting for error group managing segments... Sep 6 17:04:51.216997468 (3h36m27.256570588s) [noise=true , address=45.91.116.9:4261 , piece_id=2GEQPO5XWN57B5UAVC5WTCMIHCIYZDW4CSGMQO4EXTHK4A2RGTLQ, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.241113665 (3h36m27.280686785s) [noise=true , address=45.91.116.14:4191 , piece_id=BCPIMZPE4YU63QMM5OT3BX7EENQMXNBWXPLZEGD3ICMKUFHH7S7A, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.243256716 (3h36m27.282829836s) [noise=true , address=77.90.31.15:4699 , piece_id=G752LE6G55SFWLGWGWMOBV3OCJQYLIDLTUPAQSWPTGIKZVIHCS3Q, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.263575827 (3h36m27.303148937s) [noise=true , address=77.90.31.12:4589 , piece_id=HH5M2XL2536CRNV7AX2QGOABD2F36QQQB3FG3ZQUBHEN527WAEJA, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.270086886 (3h36m27.309660006s) [noise=true , address=45.91.116.5:4299 , piece_id=JI7R6IU7YNTTHORLJZWLILS7N4RBGXVMOSQFHSUBWJCMTC42VJEQ, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.277565416 (3h36m27.317138536s) [noise=true , address=45.91.116.8:4045 , piece_id=XZFGOL6I2FCZIR3KE7ELGE6AONVMQQZ4MFUXVQEE63MMMG76NSZA, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.293258204 (3h36m27.332831335s) [noise=true , address=45.91.116.11:4118 , piece_id=KOI23FVX43DPEK6Q653EGNV2WDHNQ3MFW6L75H2EYJ4WFCH5WVFA, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.312504762 (3h36m27.352077882s) [noise=true , address=77.90.31.10:4518 , piece_id=EAPBMGI4EX4GHIHSCNZLPCAOLVZOXH2Y6RPJ465ARFBEFJWO2CGQ, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.333552437 (3h36m27.373125557s) [noise=true , address=77.90.31.14:4662 , piece_id=Y4WBWHEFMJY45L7TU7O4DAANFNFL4TEHYZUR2M77IN7WFVV4SA5Q, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.334919302 (3h36m27.374492423s) [noise=true , address=45.91.116.11:4117 , piece_id=WNPSPOJZU7E3HB6CKM37FKG5AGLKAWOG5ZHVF5PL6O3JV7TKVIXQ, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.377061653 (3h36m27.416634773s) [noise=true , address=45.91.116.7:4009 , piece_id=3N55SSNQRUSY3LEXOYL7RIW22LDZJIFASU6EEWQ4TNZUOCBMHJ7Q, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.395711139 (3h36m27.435284259s) [noise=true , address=45.91.116.5:4297 , piece_id=4ZBL7Y5LQUIR2UJDZ3F6LBE7P4VXFUUH4GMNGFZRKU2BRSERLUSA, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.417020304 (3h36m27.456593424s) [noise=true , address=45.91.116.12:4226 , piece_id=3DS4CCYXPKYXIBLR6SH6ON26KHTIVBRR3MNVXBEOD4X2IH2TKM4A, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.417036926 (3h36m27.456610046s) [seg_pos={0 0}, upload=3145]: Segment reached optimal threshold of 13 pieces. Sep 6 17:04:51.417139733 (3h36m27.456712853s) [noise=true , address=45.91.116.11:4119 , piece_id=N4TGQKPTAXXVFOXAMQOS342SKPFHGTD2A5JW47XX73P3DGJHUXGA, seg_pos={0 0}, upload=3145]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: expected piece hash; context canceled; EOF Sep 6 17:04:51.417288377 (3h36m27.456861497s) [noise=true , address=77.90.31.21:4877 , piece_id=AG5DFWZQDLZDSJN2TAE7N5KI7BGTR2OXWPXQF46ETT55UWKNVRWA, seg_pos={0 0}, upload=3145]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Sep 6 17:04:51.417368490 (3h36m27.45694161s) [noise=true , address=77.90.31.17:4770 , piece_id=QVFTS75FYPHJTPMBWU47FSBQJOXP6NPB7NP7YHWFTL7C2JKRRPWA, seg_pos={0 0}, upload=3145]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: write tcp 188.166.225.170:33928->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:33928->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33928->77.90.31.17:4770: use of closed network connection Sep 6 17:04:51.417380063 (3h36m27.456953183s) [seg_pos={0 0}, upload=3145]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 17:04:51.417442171 (3h36m27.457015281s) [upload=3145]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 17:04:51.417446028 (3h36m27.457019149s) [upload=3145]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 17:04:51.994441372 (3h36m28.034014502s) [upload=3145]: Done uploading segments... Sep 6 17:04:51.994466570 (3h36m28.03403974s) [upload=3145]: Upload finished. err: <nil> Duration: 14.439685257s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [X...XX..XX..X.X.XX....XX.XXX.....X..X....X......X.XX.X....XX] duration: 30.002453133s FAIL (2025-09-06 17:05:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 17:04:48.767994646 (3h36m24.807567776s) [upload=3145]: Starting upload Sep 6 17:04:48.768093364 (3h36m24.807666495s) [upload=3145]: Done starting upload Sep 6 17:04:48.768203756 (3h36m24.807776876s) [upload=3145]: Uploading segments... Sep 6 17:04:48.768374243 (3h36m24.807947363s) [upload=3145]: Got next segment. Inline: false Sep 6 17:04:48.768409871 (3h36m24.807982991s) [upload=3145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 17:04:48.768414609 (3h36m24.80798773s) [upload=3145]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 17:04:49.042040494 (3h36m25.081613604s) [seg_pos={0 0}, upload=3145]: Begin upload segment... Sep 6 17:04:49.042081854 (3h36m25.081654974s) [seg_pos={0 0}, upload=3145]: Done begin upload segment. Sep 6 17:04:49.042103194 (3h36m25.081676314s) [noise=true , address=77.90.31.17:4770 , piece_id=QVFTS75FYPHJTPMBWU47FSBQJOXP6NPB7NP7YHWFTL7C2JKRRPWA, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.042130767 (3h36m25.081703877s) [noise=true , address=77.90.31.10:4518 , piece_id=EAPBMGI4EX4GHIHSCNZLPCAOLVZOXH2Y6RPJ465ARFBEFJWO2CGQ, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.042462482 (3h36m25.082035602s) [noise=true , address=77.90.31.21:4877 , piece_id=AG5DFWZQDLZDSJN2TAE7N5KI7BGTR2OXWPXQF46ETT55UWKNVRWA, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.042888778 (3h36m25.082461898s) [noise=true , address=45.91.116.7:4009 , piece_id=3N55SSNQRUSY3LEXOYL7RIW22LDZJIFASU6EEWQ4TNZUOCBMHJ7Q, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.043698517 (3h36m25.083271657s) [noise=true , address=45.91.116.5:4297 , piece_id=4ZBL7Y5LQUIR2UJDZ3F6LBE7P4VXFUUH4GMNGFZRKU2BRSERLUSA, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.044328853 (3h36m25.083902004s) [noise=true , address=77.90.31.14:4662 , piece_id=Y4WBWHEFMJY45L7TU7O4DAANFNFL4TEHYZUR2M77IN7WFVV4SA5Q, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.044920426 (3h36m25.084493576s) [noise=true , address=45.91.116.12:4226 , piece_id=3DS4CCYXPKYXIBLR6SH6ON26KHTIVBRR3MNVXBEOD4X2IH2TKM4A, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.045526355 (3h36m25.085099495s) [noise=true , address=45.91.116.11:4117 , piece_id=WNPSPOJZU7E3HB6CKM37FKG5AGLKAWOG5ZHVF5PL6O3JV7TKVIXQ, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.046803047 (3h36m25.086376167s) [noise=true , address=45.91.116.8:4045 , piece_id=XZFGOL6I2FCZIR3KE7ELGE6AONVMQQZ4MFUXVQEE63MMMG76NSZA, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.047215998 (3h36m25.086789118s) [noise=true , address=45.91.116.5:4299 , piece_id=JI7R6IU7YNTTHORLJZWLILS7N4RBGXVMOSQFHSUBWJCMTC42VJEQ, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.047482037 (3h36m25.087055157s) [noise=true , address=45.91.116.11:4119 , piece_id=N4TGQKPTAXXVFOXAMQOS342SKPFHGTD2A5JW47XX73P3DGJHUXGA, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.047778905 (3h36m25.087352025s) [noise=true , address=77.90.31.15:4699 , piece_id=G752LE6G55SFWLGWGWMOBV3OCJQYLIDLTUPAQSWPTGIKZVIHCS3Q, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.048024465 (3h36m25.087597785s) [noise=true , address=45.91.116.14:4191 , piece_id=BCPIMZPE4YU63QMM5OT3BX7EENQMXNBWXPLZEGD3ICMKUFHH7S7A, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.048291867 (3h36m25.087864987s) [noise=true , address=45.91.116.11:4118 , piece_id=KOI23FVX43DPEK6Q653EGNV2WDHNQ3MFW6L75H2EYJ4WFCH5WVFA, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.048544390 (3h36m25.08811752s) [noise=true , address=77.90.31.12:4589 , piece_id=HH5M2XL2536CRNV7AX2QGOABD2F36QQQB3FG3ZQUBHEN527WAEJA, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:49.048791262 (3h36m25.088364382s) [noise=true , address=45.91.116.9:4261 , piece_id=2GEQPO5XWN57B5UAVC5WTCMIHCIYZDW4CSGMQO4EXTHK4A2RGTLQ, seg_pos={0 0}, upload=3145]: Uploading piece... Sep 6 17:04:50.840862672 (3h36m26.880435792s) [upload=3145]: Next returned nil segment Sep 6 17:04:50.840881569 (3h36m26.880454689s) [upload=3145]: Waiting for error group managing segments... Sep 6 17:04:51.216997468 (3h36m27.256570588s) [noise=true , address=45.91.116.9:4261 , piece_id=2GEQPO5XWN57B5UAVC5WTCMIHCIYZDW4CSGMQO4EXTHK4A2RGTLQ, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.241113665 (3h36m27.280686785s) [noise=true , address=45.91.116.14:4191 , piece_id=BCPIMZPE4YU63QMM5OT3BX7EENQMXNBWXPLZEGD3ICMKUFHH7S7A, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.243256716 (3h36m27.282829836s) [noise=true , address=77.90.31.15:4699 , piece_id=G752LE6G55SFWLGWGWMOBV3OCJQYLIDLTUPAQSWPTGIKZVIHCS3Q, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.263575827 (3h36m27.303148937s) [noise=true , address=77.90.31.12:4589 , piece_id=HH5M2XL2536CRNV7AX2QGOABD2F36QQQB3FG3ZQUBHEN527WAEJA, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.270086886 (3h36m27.309660006s) [noise=true , address=45.91.116.5:4299 , piece_id=JI7R6IU7YNTTHORLJZWLILS7N4RBGXVMOSQFHSUBWJCMTC42VJEQ, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.277565416 (3h36m27.317138536s) [noise=true , address=45.91.116.8:4045 , piece_id=XZFGOL6I2FCZIR3KE7ELGE6AONVMQQZ4MFUXVQEE63MMMG76NSZA, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.293258204 (3h36m27.332831335s) [noise=true , address=45.91.116.11:4118 , piece_id=KOI23FVX43DPEK6Q653EGNV2WDHNQ3MFW6L75H2EYJ4WFCH5WVFA, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.312504762 (3h36m27.352077882s) [noise=true , address=77.90.31.10:4518 , piece_id=EAPBMGI4EX4GHIHSCNZLPCAOLVZOXH2Y6RPJ465ARFBEFJWO2CGQ, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.333552437 (3h36m27.373125557s) [noise=true , address=77.90.31.14:4662 , piece_id=Y4WBWHEFMJY45L7TU7O4DAANFNFL4TEHYZUR2M77IN7WFVV4SA5Q, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.334919302 (3h36m27.374492423s) [noise=true , address=45.91.116.11:4117 , piece_id=WNPSPOJZU7E3HB6CKM37FKG5AGLKAWOG5ZHVF5PL6O3JV7TKVIXQ, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.377061653 (3h36m27.416634773s) [noise=true , address=45.91.116.7:4009 , piece_id=3N55SSNQRUSY3LEXOYL7RIW22LDZJIFASU6EEWQ4TNZUOCBMHJ7Q, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.395711139 (3h36m27.435284259s) [noise=true , address=45.91.116.5:4297 , piece_id=4ZBL7Y5LQUIR2UJDZ3F6LBE7P4VXFUUH4GMNGFZRKU2BRSERLUSA, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.417020304 (3h36m27.456593424s) [noise=true , address=45.91.116.12:4226 , piece_id=3DS4CCYXPKYXIBLR6SH6ON26KHTIVBRR3MNVXBEOD4X2IH2TKM4A, seg_pos={0 0}, upload=3145]: Done uploading piece. err: <nil> Sep 6 17:04:51.417036926 (3h36m27.456610046s) [seg_pos={0 0}, upload=3145]: Segment reached optimal threshold of 13 pieces. Sep 6 17:04:51.417139733 (3h36m27.456712853s) [noise=true , address=45.91.116.11:4119 , piece_id=N4TGQKPTAXXVFOXAMQOS342SKPFHGTD2A5JW47XX73P3DGJHUXGA, seg_pos={0 0}, upload=3145]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: expected piece hash; context canceled; EOF Sep 6 17:04:51.417288377 (3h36m27.456861497s) [noise=true , address=77.90.31.21:4877 , piece_id=AG5DFWZQDLZDSJN2TAE7N5KI7BGTR2OXWPXQF46ETT55UWKNVRWA, seg_pos={0 0}, upload=3145]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Sep 6 17:04:51.417368490 (3h36m27.45694161s) [noise=true , address=77.90.31.17:4770 , piece_id=QVFTS75FYPHJTPMBWU47FSBQJOXP6NPB7NP7YHWFTL7C2JKRRPWA, seg_pos={0 0}, upload=3145]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: write tcp 188.166.225.170:33928->77.90.31.17:4770: use of closed network connection; write tcp 188.166.225.170:33928->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33928->77.90.31.17:4770: use of closed network connection Sep 6 17:04:51.417380063 (3h36m27.456953183s) [seg_pos={0 0}, upload=3145]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 17:04:51.417442171 (3h36m27.457015281s) [upload=3145]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 17:04:51.417446028 (3h36m27.457019149s) [upload=3145]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 17:04:51.994441372 (3h36m28.034014502s) [upload=3145]: Done uploading segments... Sep 6 17:04:51.994466570 (3h36m28.03403974s) [upload=3145]: Upload finished. err: <nil> Duration: 30.002480956s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [.X...XX..XX..X.X.XX....XX.XXX.....X..X....X......X.XX.X....X] duration: 30.002680018s FAIL (2025-09-06 17:04:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 17:04:07.599027728 (3h35m43.638600848s) [upload=3135]: Starting upload Sep 6 17:04:07.599085208 (3h35m43.638658328s) [upload=3135]: Done starting upload Sep 6 17:04:07.599173647 (3h35m43.638746757s) [upload=3135]: Uploading segments... Sep 6 17:04:07.599257918 (3h35m43.638831028s) [upload=3135]: Got next segment. Inline: false Sep 6 17:04:07.599289780 (3h35m43.6388629s) [upload=3135]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 17:04:07.599294258 (3h35m43.638867378s) [upload=3135]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 17:04:07.844717659 (3h35m43.884290779s) [seg_pos={0 0}, upload=3135]: Begin upload segment... Sep 6 17:04:07.844760580 (3h35m43.8843337s) [seg_pos={0 0}, upload=3135]: Done begin upload segment. Sep 6 17:04:07.844777764 (3h35m43.884350884s) [noise=true , address=77.90.31.13:4627 , piece_id=CJPAGNGL4D476TQEJMNDSKCO4D2T5XR6PJZEHY75SBFDUCTJFD4Q, seg_pos={0 0}, upload=3135]: Uploading piece... Sep 6 17:04:07.844788554 (3h35m43.884361674s) [noise=true , address=45.91.116.13:4153 , piece_id=6KQYOE65K5XAPVZ6I5VUYCC7FLXHCXMYWVIUPRHFSN3COCF56LYQ, seg_pos={0 0}, upload=3135]: Uploading piece... Sep 6 17:04:07.844867705 (3h35m43.884440825s) [noise=true , address=77.90.31.17:4770 , piece_id=MTCLRS3N55BX6HHEUUUSDVN3GEER47RCUYLQQM63OSM2OC43Z7CQ, seg_pos={0 0}, upload=3135]: Uploading piece... Sep 6 17:04:07.845129827 (3h35m43.884702947s) [noise=true , address=45.91.116.14:4191 , piece_id=A2D4ECYWZ36XHHMKZORIB6566JHBA3HDQGBM6AOCK4NGRA433BTA, seg_pos={0 0}, upload=3135]: Uploading piece... Sep 6 17:04:07.845425413 (3h35m43.884998533s) [noise=true , address=45.91.116.12:4226 , piece_id=JMKIIABNE2LWPFALTJYLCD6AKA3ZF4WJ5C4AXYH7S2L6CAYG63GA, seg_pos={0 0}, upload=3135]: Uploading piece... Sep 6 17:04:07.845699558 (3h35m43.885272678s) [noise=true , address=77.90.31.14:4662 , piece_id=A6TESAJ7R2PO54WXQANYODQNNQLN5ST4JWTP5EUUKQYSUUUSU6CA, seg_pos={0 0}, upload=3135]: Uploading piece... Sep 6 17:04:07.845983881 (3h35m43.885557001s) [noise=true , address=45.91.116.4:4940 , piece_id=666JUV66KPJKRYQASXKB5N3N64EAHFZRRNE345TOBGEVDO2B4FPA, seg_pos={0 0}, upload=3135]: Uploading piece... Sep 6 17:04:07.846252966 (3h35m43.885826087s) [noise=true , address=45.91.116.10:4082 , piece_id=YDYV73ZPJK5CBSCTACNUBUPOGWRBQ63A3M2PFHCWJUV3M3QJXIMQ, seg_pos={0 0}, upload=3135]: Uploading piece... Sep 6 17:04:07.846474360 (3h35m43.88604748s) [noise=true , address=45.91.116.11:4119 , piece_id=Y5OXOC5AEUO3VPMYCYSXHEHTUOKCPREDNW3PWSPXILSANIMVNKQA, seg_pos={0 0}, upload=3135]: Uploading piece... Sep 6 17:04:07.846814951 (3h35m43.886388061s) [noise=true , address=77.90.31.16:4735 , piece_id=AGHJYQFBTW24OIZWBAJVKSCYVDH7G4N5E7XTBB2NIOF3W4EGU4CA, seg_pos={0 0}, upload=3135]: Uploading piece... Sep 6 17:04:07.847128362 (3h35m43.886701482s) [noise=true , address=45.91.116.13:4154 , piece_id=7AZNAZANVOH2GPJL64GG6KTRAYXRWSMV4MFNDZ7JP5WEHIN4SKOQ, seg_pos={0 0}, upload=3135]: Uploading piece... Sep 6 17:04:07.847379742 (3h35m43.886952862s) [noise=true , address=45.91.116.13:4155 , piece_id=R66LBECDHYP464FS4OUEIEO25NWJFUQ4N2VLV3PT4PKTIT377UPA, seg_pos={0 0}, upload=3135]: Uploading piece... Sep 6 17:04:07.847637666 (3h35m43.887210787s) [noise=true , address=45.91.116.8:4047 , piece_id=JCI2UUSGDCWK2C5467GTEY5TEQXKXUF2THA7FWYNW2UAOD2QXWPQ, seg_pos={0 0}, upload=3135]: Uploading piece... Sep 6 17:04:07.847867105 (3h35m43.887440225s) [noise=true , address=45.91.116.14:4189 , piece_id=W7U52LUOBLT7Z43CBV6ZBOUEJLNPU6SDV3VBFWRVZFIH6OXHEYHA, seg_pos={0 0}, upload=3135]: Uploading piece... Sep 6 17:04:07.848083810 (3h35m43.88765693s) [noise=true , address=77.90.31.14:4663 , piece_id=SWCNSTRON4MRPKL7NJUZDSPO3PWH6ISWDI36YJMFUBRKW66KBIHA, seg_pos={0 0}, upload=3135]: Uploading piece... Sep 6 17:04:07.848296377 (3h35m43.887869487s) [noise=true , address=45.91.116.5:4298 , piece_id=PUPNHABJO7IZAADJGZJD3WEN6EMXVFCFM3C7HHJFY3R5VQSJM5BA, seg_pos={0 0}, upload=3135]: Uploading piece... Sep 6 17:04:08.102407440 (3h35m44.14198057s) [noise=true , address=77.90.31.17:4770 , piece_id=MTCLRS3N55BX6HHEUUUSDVN3GEER47RCUYLQQM63OSM2OC43Z7CQ, seg_pos={0 0}, upload=3135]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 17:04:09.690796641 (3h35m45.730369761s) [upload=3135]: Next returned nil segment Sep 6 17:04:09.690836556 (3h35m45.730409677s) [upload=3135]: Waiting for error group managing segments... Sep 6 17:04:09.998679837 (3h35m46.038252957s) [noise=true , address=45.91.116.10:4082 , piece_id=YDYV73ZPJK5CBSCTACNUBUPOGWRBQ63A3M2PFHCWJUV3M3QJXIMQ, seg_pos={0 0}, upload=3135]: Done uploading piece. err: <nil> Sep 6 17:04:10.003306650 (3h35m46.04287978s) [noise=true , address=45.91.116.13:4154 , piece_id=7AZNAZANVOH2GPJL64GG6KTRAYXRWSMV4MFNDZ7JP5WEHIN4SKOQ, seg_pos={0 0}, upload=3135]: Done uploading piece. err: <nil> Sep 6 17:04:10.016094851 (3h35m46.055667971s) [noise=true , address=45.91.116.12:4226 , piece_id=JMKIIABNE2LWPFALTJYLCD6AKA3ZF4WJ5C4AXYH7S2L6CAYG63GA, seg_pos={0 0}, upload=3135]: Done uploading piece. err: <nil> Sep 6 17:04:10.059666936 (3h35m46.099240066s) [noise=true , address=45.91.116.4:4940 , piece_id=666JUV66KPJKRYQASXKB5N3N64EAHFZRRNE345TOBGEVDO2B4FPA, seg_pos={0 0}, upload=3135]: Done uploading piece. err: <nil> Sep 6 17:04:10.080968446 (3h35m46.120541566s) [noise=true , address=77.90.31.14:4663 , piece_id=SWCNSTRON4MRPKL7NJUZDSPO3PWH6ISWDI36YJMFUBRKW66KBIHA, seg_pos={0 0}, upload=3135]: Done uploading piece. err: <nil> Sep 6 17:04:10.143741914 (3h35m46.183315034s) [noise=true , address=45.91.116.13:4155 , piece_id=R66LBECDHYP464FS4OUEIEO25NWJFUQ4N2VLV3PT4PKTIT377UPA, seg_pos={0 0}, upload=3135]: Done uploading piece. err: <nil> Sep 6 17:04:10.144717962 (3h35m46.184291082s) [noise=true , address=77.90.31.14:4662 , piece_id=A6TESAJ7R2PO54WXQANYODQNNQLN5ST4JWTP5EUUKQYSUUUSU6CA, seg_pos={0 0}, upload=3135]: Done uploading piece. err: <nil> Sep 6 17:04:10.205463829 (3h35m46.245036949s) [noise=true , address=45.91.116.14:4191 , piece_id=A2D4ECYWZ36XHHMKZORIB6566JHBA3HDQGBM6AOCK4NGRA433BTA, seg_pos={0 0}, upload=3135]: Done uploading piece. err: <nil> Sep 6 17:04:10.230953283 (3h35m46.270526403s) [noise=true , address=45.91.116.11:4119 , piece_id=Y5OXOC5AEUO3VPMYCYSXHEHTUOKCPREDNW3PWSPXILSANIMVNKQA, seg_pos={0 0}, upload=3135]: Done uploading piece. err: <nil> Sep 6 17:04:10.327796714 (3h35m46.367369834s) [noise=true , address=45.91.116.13:4153 , piece_id=6KQYOE65K5XAPVZ6I5VUYCC7FLXHCXMYWVIUPRHFSN3COCF56LYQ, seg_pos={0 0}, upload=3135]: Done uploading piece. err: <nil> Sep 6 17:04:10.422548702 (3h35m46.462121862s) [noise=true , address=45.91.116.8:4047 , piece_id=JCI2UUSGDCWK2C5467GTEY5TEQXKXUF2THA7FWYNW2UAOD2QXWPQ, seg_pos={0 0}, upload=3135]: Done uploading piece. err: <nil> Sep 6 17:04:10.544107867 (3h35m46.583681037s) [noise=true , address=77.90.31.13:4627 , piece_id=CJPAGNGL4D476TQEJMNDSKCO4D2T5XR6PJZEHY75SBFDUCTJFD4Q, seg_pos={0 0}, upload=3135]: Done uploading piece. err: <nil> Sep 6 17:04:10.611653466 (3h35m46.651226586s) [noise=true , address=45.91.116.5:4298 , piece_id=PUPNHABJO7IZAADJGZJD3WEN6EMXVFCFM3C7HHJFY3R5VQSJM5BA, seg_pos={0 0}, upload=3135]: Done uploading piece. err: <nil> Sep 6 17:04:10.611692751 (3h35m46.651265891s) [seg_pos={0 0}, upload=3135]: Segment reached optimal threshold of 13 pieces. Sep 6 17:04:10.612030036 (3h35m46.651603146s) [noise=true , address=77.90.31.16:4735 , piece_id=AGHJYQFBTW24OIZWBAJVKSCYVDH7G4N5E7XTBB2NIOF3W4EGU4CA, seg_pos={0 0}, upload=3135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h): protocol: write tcp 188.166.225.170:37140->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:37140->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37140->77.90.31.16:4735: use of closed network connection Sep 6 17:04:10.612064192 (3h35m46.651637312s) [noise=true , address=45.91.116.14:4189 , piece_id=W7U52LUOBLT7Z43CBV6ZBOUEJLNPU6SDV3VBFWRVZFIH6OXHEYHA, seg_pos={0 0}, upload=3135]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N): protocol: write tcp 188.166.225.170:46398->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:46398->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46398->45.91.116.14:4189: use of closed network connection Sep 6 17:04:10.612104379 (3h35m46.651677499s) [seg_pos={0 0}, upload=3135]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 17:04:10.612184943 (3h35m46.651758123s) [upload=3135]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 17:04:10.612192006 (3h35m46.651765126s) [upload=3135]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 17:04:10.919160802 (3h35m46.958733932s) [upload=3135]: Done uploading segments... Sep 6 17:04:10.919180089 (3h35m46.95875322s) [upload=3135]: Upload finished. err: <nil> Duration: 30.002703784s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [..X...XX..XX..X.X.XX....XX.XXX.....X..X....X......X.XX.X....] duration: 9.073433586s FAIL (2025-09-06 17:03:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 17:00:10.327043607 (3h31m46.366616727s) [upload=3074]: Starting upload Sep 6 17:00:10.327125174 (3h31m46.366698294s) [upload=3074]: Done starting upload Sep 6 17:00:10.327305728 (3h31m46.366878858s) [upload=3074]: Uploading segments... Sep 6 17:00:10.327730071 (3h31m46.367303221s) [upload=3074]: Got next segment. Inline: false Sep 6 17:00:10.327837777 (3h31m46.367410897s) [upload=3074]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 17:00:10.327857003 (3h31m46.367430123s) [upload=3074]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 17:00:10.656234576 (3h31m46.695807706s) [seg_pos={0 0}, upload=3074]: Begin upload segment... Sep 6 17:00:10.656292978 (3h31m46.695866108s) [seg_pos={0 0}, upload=3074]: Done begin upload segment. Sep 6 17:00:10.656321562 (3h31m46.695894692s) [noise=true , address=45.91.116.9:4262 , piece_id=BOWIXAXPX44RV6EW4QEFCKHFAJRVJTRDD7LCMZP7FVOLZMACPQSA, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.656357471 (3h31m46.695930601s) [noise=true , address=45.91.116.5:4299 , piece_id=YDIYWVWY26Z4ANFNY4PRG5O6M7ZT2L77NFO624BOSEXYU2AQW2BQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.656738440 (3h31m46.69631156s) [noise=true , address=45.91.116.11:4118 , piece_id=6KRA6HC46RJWHCJHODXEEHZNAF5Z6UPR7FDUXEX5QARZJ7XBTUAQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.656995702 (3h31m46.696568822s) [noise=true , address=45.91.116.12:4227 , piece_id=PXWJNK6ZBJB6RVD3MIH3DMEAXITKMYOULXIQUZZXOI4MDZ3EDJFQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.657107717 (3h31m46.696680847s) [noise=true , address=45.91.116.6:4973 , piece_id=VA6IRRQTWAPFCSQLJKVOSWLSBPLLV776ZECBW3WEKCEMX7HAVJUQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.657423230 (3h31m46.69699635s) [noise=true , address=45.91.116.7:4009 , piece_id=FBL5U3YALUTVGTOW5TGTCR2CZB5ETZQVNYVMF2YUSXHEJJ62WYIQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.657472284 (3h31m46.697045404s) [noise=true , address=45.91.116.8:4048 , piece_id=EZ55Y7EQPK3NI4TI5BSFKSOTWK5JCPTODI7QBVOHIZXEHIOGQGBQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.657732862 (3h31m46.697305982s) [noise=true , address=45.91.116.6:4974 , piece_id=3XY6VXEDWJHKEDMABLUQPT7QLRCCAVUXBWP5NUY6TYK62D3Q4F4Q, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.657782457 (3h31m46.697355577s) [noise=true , address=45.91.116.14:4190 , piece_id=NAOKDVREYPW3CJ2V6MGCONKP5KBT4B5OYFHTPYTPLOJ2TZ4KHXLQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.658078374 (3h31m46.697651494s) [noise=true , address=45.91.116.9:4263 , piece_id=EQEHK6BNFSLTW4CDLQ7SLZSGDZAKQ2BAENQBS5Z7PZYAK6KHA37A, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.658093553 (3h31m46.697666673s) [noise=true , address=45.91.116.14:4189 , piece_id=NTPS6BT7GHIFISATSDRKCXS5X2UMX6JX7RJWA5DKIONOJR5SLBCA, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.658359441 (3h31m46.697932561s) [noise=true , address=45.91.116.5:4297 , piece_id=ILJLOG3MTVZSB6Y7O6KK3Q7USTKPUTNHI56ZROCCMRAZHT2CVDTQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.659011078 (3h31m46.698584209s) [noise=true , address=77.90.31.16:4734 , piece_id=N2J6PBZLFSBWR4NFSHKV3ED5GWAUOYNHGT4K3GLXVTEXD26BY3KA, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.660164024 (3h31m46.699737155s) [noise=true , address=45.91.116.10:4081 , piece_id=AOCNU7BWALQFOFDOCD5FUOEUJZVNY7G7UEJEDX52GMQ44UPZKREA, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.660643432 (3h31m46.700216552s) [noise=true , address=45.91.116.11:4117 , piece_id=B5IGTWIXTRWCMBBFOZSBZF2I45DVBMO7E4CFDYMBQVYYMD5WXRFQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.660725028 (3h31m46.700298148s) [noise=true , address=77.90.31.17:4771 , piece_id=5FTW5FEE6MNGPBNDUG23XLXRDT4PIRLA75RP522JL52DPV4ZNIXA, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:12.389402996 (3h31m48.428976116s) [upload=3074]: Next returned nil segment Sep 6 17:00:12.389419336 (3h31m48.428992456s) [upload=3074]: Waiting for error group managing segments... Sep 6 17:00:12.774471415 (3h31m48.814044545s) [noise=true , address=45.91.116.5:4299 , piece_id=YDIYWVWY26Z4ANFNY4PRG5O6M7ZT2L77NFO624BOSEXYU2AQW2BQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.806312743 (3h31m48.845885873s) [noise=true , address=45.91.116.11:4117 , piece_id=B5IGTWIXTRWCMBBFOZSBZF2I45DVBMO7E4CFDYMBQVYYMD5WXRFQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.820412504 (3h31m48.859985634s) [noise=true , address=77.90.31.16:4734 , piece_id=N2J6PBZLFSBWR4NFSHKV3ED5GWAUOYNHGT4K3GLXVTEXD26BY3KA, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.897993805 (3h31m48.937566925s) [noise=true , address=45.91.116.6:4974 , piece_id=3XY6VXEDWJHKEDMABLUQPT7QLRCCAVUXBWP5NUY6TYK62D3Q4F4Q, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.900868406 (3h31m48.940441526s) [noise=true , address=77.90.31.17:4771 , piece_id=5FTW5FEE6MNGPBNDUG23XLXRDT4PIRLA75RP522JL52DPV4ZNIXA, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.908303423 (3h31m48.947876553s) [noise=true , address=45.91.116.14:4190 , piece_id=NAOKDVREYPW3CJ2V6MGCONKP5KBT4B5OYFHTPYTPLOJ2TZ4KHXLQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.908863635 (3h31m48.948436755s) [noise=true , address=45.91.116.12:4227 , piece_id=PXWJNK6ZBJB6RVD3MIH3DMEAXITKMYOULXIQUZZXOI4MDZ3EDJFQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.928649726 (3h31m48.968222846s) [noise=true , address=45.91.116.9:4263 , piece_id=EQEHK6BNFSLTW4CDLQ7SLZSGDZAKQ2BAENQBS5Z7PZYAK6KHA37A, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.990396268 (3h31m49.029969388s) [noise=true , address=45.91.116.5:4297 , piece_id=ILJLOG3MTVZSB6Y7O6KK3Q7USTKPUTNHI56ZROCCMRAZHT2CVDTQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:13.097332763 (3h31m49.136905883s) [noise=true , address=45.91.116.6:4973 , piece_id=VA6IRRQTWAPFCSQLJKVOSWLSBPLLV776ZECBW3WEKCEMX7HAVJUQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:13.285622427 (3h31m49.325195547s) [noise=true , address=45.91.116.10:4081 , piece_id=AOCNU7BWALQFOFDOCD5FUOEUJZVNY7G7UEJEDX52GMQ44UPZKREA, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:13.415767089 (3h31m49.455340209s) [noise=true , address=45.91.116.14:4189 , piece_id=NTPS6BT7GHIFISATSDRKCXS5X2UMX6JX7RJWA5DKIONOJR5SLBCA, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:13.529442608 (3h31m49.569015728s) [noise=true , address=45.91.116.8:4048 , piece_id=EZ55Y7EQPK3NI4TI5BSFKSOTWK5JCPTODI7QBVOHIZXEHIOGQGBQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:13.529481111 (3h31m49.569054231s) [seg_pos={0 0}, upload=3074]: Segment reached optimal threshold of 13 pieces. Sep 6 17:00:13.529729416 (3h31m49.569302537s) [noise=true , address=45.91.116.7:4009 , piece_id=FBL5U3YALUTVGTOW5TGTCR2CZB5ETZQVNYVMF2YUSXHEJJ62WYIQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:42532->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:42532->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42532->45.91.116.7:4009: use of closed network connection Sep 6 17:00:13.529784843 (3h31m49.569357963s) [noise=true , address=45.91.116.11:4118 , piece_id=6KRA6HC46RJWHCJHODXEEHZNAF5Z6UPR7FDUXEX5QARZJ7XBTUAQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 6 17:00:13.529833165 (3h31m49.569406285s) [noise=true , address=45.91.116.9:4262 , piece_id=BOWIXAXPX44RV6EW4QEFCKHFAJRVJTRDD7LCMZP7FVOLZMACPQSA, seg_pos={0 0}, upload=3074]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Sep 6 17:00:13.529855438 (3h31m49.569428558s) [seg_pos={0 0}, upload=3074]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 17:00:13.529928617 (3h31m49.569501737s) [upload=3074]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 17:00:13.529932845 (3h31m49.569505965s) [upload=3074]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 17:00:13.855937868 (3h31m49.895510988s) [upload=3074]: Done uploading segments... Sep 6 17:00:13.855956062 (3h31m49.895529192s) [upload=3074]: Upload finished. err: <nil> Duration: 9.073457039s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [X..X...XX..XX..X.X.XX....XX.XXX.....X..X....X......X.XX.X...] duration: 6.348410664s FAIL (2025-09-06 17:02:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 17:00:10.327043607 (3h31m46.366616727s) [upload=3074]: Starting upload Sep 6 17:00:10.327125174 (3h31m46.366698294s) [upload=3074]: Done starting upload Sep 6 17:00:10.327305728 (3h31m46.366878858s) [upload=3074]: Uploading segments... Sep 6 17:00:10.327730071 (3h31m46.367303221s) [upload=3074]: Got next segment. Inline: false Sep 6 17:00:10.327837777 (3h31m46.367410897s) [upload=3074]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 17:00:10.327857003 (3h31m46.367430123s) [upload=3074]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 17:00:10.656234576 (3h31m46.695807706s) [seg_pos={0 0}, upload=3074]: Begin upload segment... Sep 6 17:00:10.656292978 (3h31m46.695866108s) [seg_pos={0 0}, upload=3074]: Done begin upload segment. Sep 6 17:00:10.656321562 (3h31m46.695894692s) [noise=true , address=45.91.116.9:4262 , piece_id=BOWIXAXPX44RV6EW4QEFCKHFAJRVJTRDD7LCMZP7FVOLZMACPQSA, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.656357471 (3h31m46.695930601s) [noise=true , address=45.91.116.5:4299 , piece_id=YDIYWVWY26Z4ANFNY4PRG5O6M7ZT2L77NFO624BOSEXYU2AQW2BQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.656738440 (3h31m46.69631156s) [noise=true , address=45.91.116.11:4118 , piece_id=6KRA6HC46RJWHCJHODXEEHZNAF5Z6UPR7FDUXEX5QARZJ7XBTUAQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.656995702 (3h31m46.696568822s) [noise=true , address=45.91.116.12:4227 , piece_id=PXWJNK6ZBJB6RVD3MIH3DMEAXITKMYOULXIQUZZXOI4MDZ3EDJFQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.657107717 (3h31m46.696680847s) [noise=true , address=45.91.116.6:4973 , piece_id=VA6IRRQTWAPFCSQLJKVOSWLSBPLLV776ZECBW3WEKCEMX7HAVJUQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.657423230 (3h31m46.69699635s) [noise=true , address=45.91.116.7:4009 , piece_id=FBL5U3YALUTVGTOW5TGTCR2CZB5ETZQVNYVMF2YUSXHEJJ62WYIQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.657472284 (3h31m46.697045404s) [noise=true , address=45.91.116.8:4048 , piece_id=EZ55Y7EQPK3NI4TI5BSFKSOTWK5JCPTODI7QBVOHIZXEHIOGQGBQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.657732862 (3h31m46.697305982s) [noise=true , address=45.91.116.6:4974 , piece_id=3XY6VXEDWJHKEDMABLUQPT7QLRCCAVUXBWP5NUY6TYK62D3Q4F4Q, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.657782457 (3h31m46.697355577s) [noise=true , address=45.91.116.14:4190 , piece_id=NAOKDVREYPW3CJ2V6MGCONKP5KBT4B5OYFHTPYTPLOJ2TZ4KHXLQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.658078374 (3h31m46.697651494s) [noise=true , address=45.91.116.9:4263 , piece_id=EQEHK6BNFSLTW4CDLQ7SLZSGDZAKQ2BAENQBS5Z7PZYAK6KHA37A, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.658093553 (3h31m46.697666673s) [noise=true , address=45.91.116.14:4189 , piece_id=NTPS6BT7GHIFISATSDRKCXS5X2UMX6JX7RJWA5DKIONOJR5SLBCA, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.658359441 (3h31m46.697932561s) [noise=true , address=45.91.116.5:4297 , piece_id=ILJLOG3MTVZSB6Y7O6KK3Q7USTKPUTNHI56ZROCCMRAZHT2CVDTQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.659011078 (3h31m46.698584209s) [noise=true , address=77.90.31.16:4734 , piece_id=N2J6PBZLFSBWR4NFSHKV3ED5GWAUOYNHGT4K3GLXVTEXD26BY3KA, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.660164024 (3h31m46.699737155s) [noise=true , address=45.91.116.10:4081 , piece_id=AOCNU7BWALQFOFDOCD5FUOEUJZVNY7G7UEJEDX52GMQ44UPZKREA, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.660643432 (3h31m46.700216552s) [noise=true , address=45.91.116.11:4117 , piece_id=B5IGTWIXTRWCMBBFOZSBZF2I45DVBMO7E4CFDYMBQVYYMD5WXRFQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.660725028 (3h31m46.700298148s) [noise=true , address=77.90.31.17:4771 , piece_id=5FTW5FEE6MNGPBNDUG23XLXRDT4PIRLA75RP522JL52DPV4ZNIXA, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:12.389402996 (3h31m48.428976116s) [upload=3074]: Next returned nil segment Sep 6 17:00:12.389419336 (3h31m48.428992456s) [upload=3074]: Waiting for error group managing segments... Sep 6 17:00:12.774471415 (3h31m48.814044545s) [noise=true , address=45.91.116.5:4299 , piece_id=YDIYWVWY26Z4ANFNY4PRG5O6M7ZT2L77NFO624BOSEXYU2AQW2BQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.806312743 (3h31m48.845885873s) [noise=true , address=45.91.116.11:4117 , piece_id=B5IGTWIXTRWCMBBFOZSBZF2I45DVBMO7E4CFDYMBQVYYMD5WXRFQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.820412504 (3h31m48.859985634s) [noise=true , address=77.90.31.16:4734 , piece_id=N2J6PBZLFSBWR4NFSHKV3ED5GWAUOYNHGT4K3GLXVTEXD26BY3KA, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.897993805 (3h31m48.937566925s) [noise=true , address=45.91.116.6:4974 , piece_id=3XY6VXEDWJHKEDMABLUQPT7QLRCCAVUXBWP5NUY6TYK62D3Q4F4Q, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.900868406 (3h31m48.940441526s) [noise=true , address=77.90.31.17:4771 , piece_id=5FTW5FEE6MNGPBNDUG23XLXRDT4PIRLA75RP522JL52DPV4ZNIXA, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.908303423 (3h31m48.947876553s) [noise=true , address=45.91.116.14:4190 , piece_id=NAOKDVREYPW3CJ2V6MGCONKP5KBT4B5OYFHTPYTPLOJ2TZ4KHXLQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.908863635 (3h31m48.948436755s) [noise=true , address=45.91.116.12:4227 , piece_id=PXWJNK6ZBJB6RVD3MIH3DMEAXITKMYOULXIQUZZXOI4MDZ3EDJFQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.928649726 (3h31m48.968222846s) [noise=true , address=45.91.116.9:4263 , piece_id=EQEHK6BNFSLTW4CDLQ7SLZSGDZAKQ2BAENQBS5Z7PZYAK6KHA37A, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.990396268 (3h31m49.029969388s) [noise=true , address=45.91.116.5:4297 , piece_id=ILJLOG3MTVZSB6Y7O6KK3Q7USTKPUTNHI56ZROCCMRAZHT2CVDTQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:13.097332763 (3h31m49.136905883s) [noise=true , address=45.91.116.6:4973 , piece_id=VA6IRRQTWAPFCSQLJKVOSWLSBPLLV776ZECBW3WEKCEMX7HAVJUQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:13.285622427 (3h31m49.325195547s) [noise=true , address=45.91.116.10:4081 , piece_id=AOCNU7BWALQFOFDOCD5FUOEUJZVNY7G7UEJEDX52GMQ44UPZKREA, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:13.415767089 (3h31m49.455340209s) [noise=true , address=45.91.116.14:4189 , piece_id=NTPS6BT7GHIFISATSDRKCXS5X2UMX6JX7RJWA5DKIONOJR5SLBCA, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:13.529442608 (3h31m49.569015728s) [noise=true , address=45.91.116.8:4048 , piece_id=EZ55Y7EQPK3NI4TI5BSFKSOTWK5JCPTODI7QBVOHIZXEHIOGQGBQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:13.529481111 (3h31m49.569054231s) [seg_pos={0 0}, upload=3074]: Segment reached optimal threshold of 13 pieces. Sep 6 17:00:13.529729416 (3h31m49.569302537s) [noise=true , address=45.91.116.7:4009 , piece_id=FBL5U3YALUTVGTOW5TGTCR2CZB5ETZQVNYVMF2YUSXHEJJ62WYIQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:42532->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:42532->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42532->45.91.116.7:4009: use of closed network connection Sep 6 17:00:13.529784843 (3h31m49.569357963s) [noise=true , address=45.91.116.11:4118 , piece_id=6KRA6HC46RJWHCJHODXEEHZNAF5Z6UPR7FDUXEX5QARZJ7XBTUAQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 6 17:00:13.529833165 (3h31m49.569406285s) [noise=true , address=45.91.116.9:4262 , piece_id=BOWIXAXPX44RV6EW4QEFCKHFAJRVJTRDD7LCMZP7FVOLZMACPQSA, seg_pos={0 0}, upload=3074]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Sep 6 17:00:13.529855438 (3h31m49.569428558s) [seg_pos={0 0}, upload=3074]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 17:00:13.529928617 (3h31m49.569501737s) [upload=3074]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 17:00:13.529932845 (3h31m49.569505965s) [upload=3074]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 17:00:13.855937868 (3h31m49.895510988s) [upload=3074]: Done uploading segments... Sep 6 17:00:13.855956062 (3h31m49.895529192s) [upload=3074]: Upload finished. err: <nil> Duration: 6.348434519s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [.X..X...XX..XX..X.X.XX....XX.XXX.....X..X....X......X.XX.X..] duration: 18.818696438s FAIL (2025-09-06 17:01:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 17:00:10.327043607 (3h31m46.366616727s) [upload=3074]: Starting upload Sep 6 17:00:10.327125174 (3h31m46.366698294s) [upload=3074]: Done starting upload Sep 6 17:00:10.327305728 (3h31m46.366878858s) [upload=3074]: Uploading segments... Sep 6 17:00:10.327730071 (3h31m46.367303221s) [upload=3074]: Got next segment. Inline: false Sep 6 17:00:10.327837777 (3h31m46.367410897s) [upload=3074]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 17:00:10.327857003 (3h31m46.367430123s) [upload=3074]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 17:00:10.656234576 (3h31m46.695807706s) [seg_pos={0 0}, upload=3074]: Begin upload segment... Sep 6 17:00:10.656292978 (3h31m46.695866108s) [seg_pos={0 0}, upload=3074]: Done begin upload segment. Sep 6 17:00:10.656321562 (3h31m46.695894692s) [noise=true , address=45.91.116.9:4262 , piece_id=BOWIXAXPX44RV6EW4QEFCKHFAJRVJTRDD7LCMZP7FVOLZMACPQSA, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.656357471 (3h31m46.695930601s) [noise=true , address=45.91.116.5:4299 , piece_id=YDIYWVWY26Z4ANFNY4PRG5O6M7ZT2L77NFO624BOSEXYU2AQW2BQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.656738440 (3h31m46.69631156s) [noise=true , address=45.91.116.11:4118 , piece_id=6KRA6HC46RJWHCJHODXEEHZNAF5Z6UPR7FDUXEX5QARZJ7XBTUAQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.656995702 (3h31m46.696568822s) [noise=true , address=45.91.116.12:4227 , piece_id=PXWJNK6ZBJB6RVD3MIH3DMEAXITKMYOULXIQUZZXOI4MDZ3EDJFQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.657107717 (3h31m46.696680847s) [noise=true , address=45.91.116.6:4973 , piece_id=VA6IRRQTWAPFCSQLJKVOSWLSBPLLV776ZECBW3WEKCEMX7HAVJUQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.657423230 (3h31m46.69699635s) [noise=true , address=45.91.116.7:4009 , piece_id=FBL5U3YALUTVGTOW5TGTCR2CZB5ETZQVNYVMF2YUSXHEJJ62WYIQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.657472284 (3h31m46.697045404s) [noise=true , address=45.91.116.8:4048 , piece_id=EZ55Y7EQPK3NI4TI5BSFKSOTWK5JCPTODI7QBVOHIZXEHIOGQGBQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.657732862 (3h31m46.697305982s) [noise=true , address=45.91.116.6:4974 , piece_id=3XY6VXEDWJHKEDMABLUQPT7QLRCCAVUXBWP5NUY6TYK62D3Q4F4Q, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.657782457 (3h31m46.697355577s) [noise=true , address=45.91.116.14:4190 , piece_id=NAOKDVREYPW3CJ2V6MGCONKP5KBT4B5OYFHTPYTPLOJ2TZ4KHXLQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.658078374 (3h31m46.697651494s) [noise=true , address=45.91.116.9:4263 , piece_id=EQEHK6BNFSLTW4CDLQ7SLZSGDZAKQ2BAENQBS5Z7PZYAK6KHA37A, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.658093553 (3h31m46.697666673s) [noise=true , address=45.91.116.14:4189 , piece_id=NTPS6BT7GHIFISATSDRKCXS5X2UMX6JX7RJWA5DKIONOJR5SLBCA, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.658359441 (3h31m46.697932561s) [noise=true , address=45.91.116.5:4297 , piece_id=ILJLOG3MTVZSB6Y7O6KK3Q7USTKPUTNHI56ZROCCMRAZHT2CVDTQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.659011078 (3h31m46.698584209s) [noise=true , address=77.90.31.16:4734 , piece_id=N2J6PBZLFSBWR4NFSHKV3ED5GWAUOYNHGT4K3GLXVTEXD26BY3KA, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.660164024 (3h31m46.699737155s) [noise=true , address=45.91.116.10:4081 , piece_id=AOCNU7BWALQFOFDOCD5FUOEUJZVNY7G7UEJEDX52GMQ44UPZKREA, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.660643432 (3h31m46.700216552s) [noise=true , address=45.91.116.11:4117 , piece_id=B5IGTWIXTRWCMBBFOZSBZF2I45DVBMO7E4CFDYMBQVYYMD5WXRFQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.660725028 (3h31m46.700298148s) [noise=true , address=77.90.31.17:4771 , piece_id=5FTW5FEE6MNGPBNDUG23XLXRDT4PIRLA75RP522JL52DPV4ZNIXA, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:12.389402996 (3h31m48.428976116s) [upload=3074]: Next returned nil segment Sep 6 17:00:12.389419336 (3h31m48.428992456s) [upload=3074]: Waiting for error group managing segments... Sep 6 17:00:12.774471415 (3h31m48.814044545s) [noise=true , address=45.91.116.5:4299 , piece_id=YDIYWVWY26Z4ANFNY4PRG5O6M7ZT2L77NFO624BOSEXYU2AQW2BQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.806312743 (3h31m48.845885873s) [noise=true , address=45.91.116.11:4117 , piece_id=B5IGTWIXTRWCMBBFOZSBZF2I45DVBMO7E4CFDYMBQVYYMD5WXRFQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.820412504 (3h31m48.859985634s) [noise=true , address=77.90.31.16:4734 , piece_id=N2J6PBZLFSBWR4NFSHKV3ED5GWAUOYNHGT4K3GLXVTEXD26BY3KA, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.897993805 (3h31m48.937566925s) [noise=true , address=45.91.116.6:4974 , piece_id=3XY6VXEDWJHKEDMABLUQPT7QLRCCAVUXBWP5NUY6TYK62D3Q4F4Q, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.900868406 (3h31m48.940441526s) [noise=true , address=77.90.31.17:4771 , piece_id=5FTW5FEE6MNGPBNDUG23XLXRDT4PIRLA75RP522JL52DPV4ZNIXA, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.908303423 (3h31m48.947876553s) [noise=true , address=45.91.116.14:4190 , piece_id=NAOKDVREYPW3CJ2V6MGCONKP5KBT4B5OYFHTPYTPLOJ2TZ4KHXLQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.908863635 (3h31m48.948436755s) [noise=true , address=45.91.116.12:4227 , piece_id=PXWJNK6ZBJB6RVD3MIH3DMEAXITKMYOULXIQUZZXOI4MDZ3EDJFQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.928649726 (3h31m48.968222846s) [noise=true , address=45.91.116.9:4263 , piece_id=EQEHK6BNFSLTW4CDLQ7SLZSGDZAKQ2BAENQBS5Z7PZYAK6KHA37A, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.990396268 (3h31m49.029969388s) [noise=true , address=45.91.116.5:4297 , piece_id=ILJLOG3MTVZSB6Y7O6KK3Q7USTKPUTNHI56ZROCCMRAZHT2CVDTQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:13.097332763 (3h31m49.136905883s) [noise=true , address=45.91.116.6:4973 , piece_id=VA6IRRQTWAPFCSQLJKVOSWLSBPLLV776ZECBW3WEKCEMX7HAVJUQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:13.285622427 (3h31m49.325195547s) [noise=true , address=45.91.116.10:4081 , piece_id=AOCNU7BWALQFOFDOCD5FUOEUJZVNY7G7UEJEDX52GMQ44UPZKREA, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:13.415767089 (3h31m49.455340209s) [noise=true , address=45.91.116.14:4189 , piece_id=NTPS6BT7GHIFISATSDRKCXS5X2UMX6JX7RJWA5DKIONOJR5SLBCA, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:13.529442608 (3h31m49.569015728s) [noise=true , address=45.91.116.8:4048 , piece_id=EZ55Y7EQPK3NI4TI5BSFKSOTWK5JCPTODI7QBVOHIZXEHIOGQGBQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:13.529481111 (3h31m49.569054231s) [seg_pos={0 0}, upload=3074]: Segment reached optimal threshold of 13 pieces. Sep 6 17:00:13.529729416 (3h31m49.569302537s) [noise=true , address=45.91.116.7:4009 , piece_id=FBL5U3YALUTVGTOW5TGTCR2CZB5ETZQVNYVMF2YUSXHEJJ62WYIQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:42532->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:42532->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42532->45.91.116.7:4009: use of closed network connection Sep 6 17:00:13.529784843 (3h31m49.569357963s) [noise=true , address=45.91.116.11:4118 , piece_id=6KRA6HC46RJWHCJHODXEEHZNAF5Z6UPR7FDUXEX5QARZJ7XBTUAQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 6 17:00:13.529833165 (3h31m49.569406285s) [noise=true , address=45.91.116.9:4262 , piece_id=BOWIXAXPX44RV6EW4QEFCKHFAJRVJTRDD7LCMZP7FVOLZMACPQSA, seg_pos={0 0}, upload=3074]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Sep 6 17:00:13.529855438 (3h31m49.569428558s) [seg_pos={0 0}, upload=3074]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 17:00:13.529928617 (3h31m49.569501737s) [upload=3074]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 17:00:13.529932845 (3h31m49.569505965s) [upload=3074]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 17:00:13.855937868 (3h31m49.895510988s) [upload=3074]: Done uploading segments... Sep 6 17:00:13.855956062 (3h31m49.895529192s) [upload=3074]: Upload finished. err: <nil> Duration: 18.818722979s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [..X..X...XX..XX..X.X.XX....XX.XXX.....X..X....X......X.XX.X.] duration: 7.992515283s FAIL (2025-09-06 17:00:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 17:00:10.327043607 (3h31m46.366616727s) [upload=3074]: Starting upload Sep 6 17:00:10.327125174 (3h31m46.366698294s) [upload=3074]: Done starting upload Sep 6 17:00:10.327305728 (3h31m46.366878858s) [upload=3074]: Uploading segments... Sep 6 17:00:10.327730071 (3h31m46.367303221s) [upload=3074]: Got next segment. Inline: false Sep 6 17:00:10.327837777 (3h31m46.367410897s) [upload=3074]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 17:00:10.327857003 (3h31m46.367430123s) [upload=3074]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 17:00:10.656234576 (3h31m46.695807706s) [seg_pos={0 0}, upload=3074]: Begin upload segment... Sep 6 17:00:10.656292978 (3h31m46.695866108s) [seg_pos={0 0}, upload=3074]: Done begin upload segment. Sep 6 17:00:10.656321562 (3h31m46.695894692s) [noise=true , address=45.91.116.9:4262 , piece_id=BOWIXAXPX44RV6EW4QEFCKHFAJRVJTRDD7LCMZP7FVOLZMACPQSA, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.656357471 (3h31m46.695930601s) [noise=true , address=45.91.116.5:4299 , piece_id=YDIYWVWY26Z4ANFNY4PRG5O6M7ZT2L77NFO624BOSEXYU2AQW2BQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.656738440 (3h31m46.69631156s) [noise=true , address=45.91.116.11:4118 , piece_id=6KRA6HC46RJWHCJHODXEEHZNAF5Z6UPR7FDUXEX5QARZJ7XBTUAQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.656995702 (3h31m46.696568822s) [noise=true , address=45.91.116.12:4227 , piece_id=PXWJNK6ZBJB6RVD3MIH3DMEAXITKMYOULXIQUZZXOI4MDZ3EDJFQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.657107717 (3h31m46.696680847s) [noise=true , address=45.91.116.6:4973 , piece_id=VA6IRRQTWAPFCSQLJKVOSWLSBPLLV776ZECBW3WEKCEMX7HAVJUQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.657423230 (3h31m46.69699635s) [noise=true , address=45.91.116.7:4009 , piece_id=FBL5U3YALUTVGTOW5TGTCR2CZB5ETZQVNYVMF2YUSXHEJJ62WYIQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.657472284 (3h31m46.697045404s) [noise=true , address=45.91.116.8:4048 , piece_id=EZ55Y7EQPK3NI4TI5BSFKSOTWK5JCPTODI7QBVOHIZXEHIOGQGBQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.657732862 (3h31m46.697305982s) [noise=true , address=45.91.116.6:4974 , piece_id=3XY6VXEDWJHKEDMABLUQPT7QLRCCAVUXBWP5NUY6TYK62D3Q4F4Q, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.657782457 (3h31m46.697355577s) [noise=true , address=45.91.116.14:4190 , piece_id=NAOKDVREYPW3CJ2V6MGCONKP5KBT4B5OYFHTPYTPLOJ2TZ4KHXLQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.658078374 (3h31m46.697651494s) [noise=true , address=45.91.116.9:4263 , piece_id=EQEHK6BNFSLTW4CDLQ7SLZSGDZAKQ2BAENQBS5Z7PZYAK6KHA37A, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.658093553 (3h31m46.697666673s) [noise=true , address=45.91.116.14:4189 , piece_id=NTPS6BT7GHIFISATSDRKCXS5X2UMX6JX7RJWA5DKIONOJR5SLBCA, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.658359441 (3h31m46.697932561s) [noise=true , address=45.91.116.5:4297 , piece_id=ILJLOG3MTVZSB6Y7O6KK3Q7USTKPUTNHI56ZROCCMRAZHT2CVDTQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.659011078 (3h31m46.698584209s) [noise=true , address=77.90.31.16:4734 , piece_id=N2J6PBZLFSBWR4NFSHKV3ED5GWAUOYNHGT4K3GLXVTEXD26BY3KA, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.660164024 (3h31m46.699737155s) [noise=true , address=45.91.116.10:4081 , piece_id=AOCNU7BWALQFOFDOCD5FUOEUJZVNY7G7UEJEDX52GMQ44UPZKREA, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.660643432 (3h31m46.700216552s) [noise=true , address=45.91.116.11:4117 , piece_id=B5IGTWIXTRWCMBBFOZSBZF2I45DVBMO7E4CFDYMBQVYYMD5WXRFQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.660725028 (3h31m46.700298148s) [noise=true , address=77.90.31.17:4771 , piece_id=5FTW5FEE6MNGPBNDUG23XLXRDT4PIRLA75RP522JL52DPV4ZNIXA, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:12.389402996 (3h31m48.428976116s) [upload=3074]: Next returned nil segment Sep 6 17:00:12.389419336 (3h31m48.428992456s) [upload=3074]: Waiting for error group managing segments... Sep 6 17:00:12.774471415 (3h31m48.814044545s) [noise=true , address=45.91.116.5:4299 , piece_id=YDIYWVWY26Z4ANFNY4PRG5O6M7ZT2L77NFO624BOSEXYU2AQW2BQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.806312743 (3h31m48.845885873s) [noise=true , address=45.91.116.11:4117 , piece_id=B5IGTWIXTRWCMBBFOZSBZF2I45DVBMO7E4CFDYMBQVYYMD5WXRFQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.820412504 (3h31m48.859985634s) [noise=true , address=77.90.31.16:4734 , piece_id=N2J6PBZLFSBWR4NFSHKV3ED5GWAUOYNHGT4K3GLXVTEXD26BY3KA, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.897993805 (3h31m48.937566925s) [noise=true , address=45.91.116.6:4974 , piece_id=3XY6VXEDWJHKEDMABLUQPT7QLRCCAVUXBWP5NUY6TYK62D3Q4F4Q, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.900868406 (3h31m48.940441526s) [noise=true , address=77.90.31.17:4771 , piece_id=5FTW5FEE6MNGPBNDUG23XLXRDT4PIRLA75RP522JL52DPV4ZNIXA, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.908303423 (3h31m48.947876553s) [noise=true , address=45.91.116.14:4190 , piece_id=NAOKDVREYPW3CJ2V6MGCONKP5KBT4B5OYFHTPYTPLOJ2TZ4KHXLQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.908863635 (3h31m48.948436755s) [noise=true , address=45.91.116.12:4227 , piece_id=PXWJNK6ZBJB6RVD3MIH3DMEAXITKMYOULXIQUZZXOI4MDZ3EDJFQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.928649726 (3h31m48.968222846s) [noise=true , address=45.91.116.9:4263 , piece_id=EQEHK6BNFSLTW4CDLQ7SLZSGDZAKQ2BAENQBS5Z7PZYAK6KHA37A, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.990396268 (3h31m49.029969388s) [noise=true , address=45.91.116.5:4297 , piece_id=ILJLOG3MTVZSB6Y7O6KK3Q7USTKPUTNHI56ZROCCMRAZHT2CVDTQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:13.097332763 (3h31m49.136905883s) [noise=true , address=45.91.116.6:4973 , piece_id=VA6IRRQTWAPFCSQLJKVOSWLSBPLLV776ZECBW3WEKCEMX7HAVJUQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:13.285622427 (3h31m49.325195547s) [noise=true , address=45.91.116.10:4081 , piece_id=AOCNU7BWALQFOFDOCD5FUOEUJZVNY7G7UEJEDX52GMQ44UPZKREA, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:13.415767089 (3h31m49.455340209s) [noise=true , address=45.91.116.14:4189 , piece_id=NTPS6BT7GHIFISATSDRKCXS5X2UMX6JX7RJWA5DKIONOJR5SLBCA, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:13.529442608 (3h31m49.569015728s) [noise=true , address=45.91.116.8:4048 , piece_id=EZ55Y7EQPK3NI4TI5BSFKSOTWK5JCPTODI7QBVOHIZXEHIOGQGBQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:13.529481111 (3h31m49.569054231s) [seg_pos={0 0}, upload=3074]: Segment reached optimal threshold of 13 pieces. Sep 6 17:00:13.529729416 (3h31m49.569302537s) [noise=true , address=45.91.116.7:4009 , piece_id=FBL5U3YALUTVGTOW5TGTCR2CZB5ETZQVNYVMF2YUSXHEJJ62WYIQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:42532->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:42532->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42532->45.91.116.7:4009: use of closed network connection Sep 6 17:00:13.529784843 (3h31m49.569357963s) [noise=true , address=45.91.116.11:4118 , piece_id=6KRA6HC46RJWHCJHODXEEHZNAF5Z6UPR7FDUXEX5QARZJ7XBTUAQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 6 17:00:13.529833165 (3h31m49.569406285s) [noise=true , address=45.91.116.9:4262 , piece_id=BOWIXAXPX44RV6EW4QEFCKHFAJRVJTRDD7LCMZP7FVOLZMACPQSA, seg_pos={0 0}, upload=3074]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Sep 6 17:00:13.529855438 (3h31m49.569428558s) [seg_pos={0 0}, upload=3074]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 17:00:13.529928617 (3h31m49.569501737s) [upload=3074]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 17:00:13.529932845 (3h31m49.569505965s) [upload=3074]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 17:00:13.855937868 (3h31m49.895510988s) [upload=3074]: Done uploading segments... Sep 6 17:00:13.855956062 (3h31m49.895529192s) [upload=3074]: Upload finished. err: <nil> Duration: 7.992535572s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [X..X..X...XX..XX..X.X.XX....XX.XXX.....X..X....X......X.XX.X] duration: 30.002505647s FAIL (2025-09-06 17:00:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 17:00:10.327043607 (3h31m46.366616727s) [upload=3074]: Starting upload Sep 6 17:00:10.327125174 (3h31m46.366698294s) [upload=3074]: Done starting upload Sep 6 17:00:10.327305728 (3h31m46.366878858s) [upload=3074]: Uploading segments... Sep 6 17:00:10.327730071 (3h31m46.367303221s) [upload=3074]: Got next segment. Inline: false Sep 6 17:00:10.327837777 (3h31m46.367410897s) [upload=3074]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 17:00:10.327857003 (3h31m46.367430123s) [upload=3074]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 17:00:10.656234576 (3h31m46.695807706s) [seg_pos={0 0}, upload=3074]: Begin upload segment... Sep 6 17:00:10.656292978 (3h31m46.695866108s) [seg_pos={0 0}, upload=3074]: Done begin upload segment. Sep 6 17:00:10.656321562 (3h31m46.695894692s) [noise=true , address=45.91.116.9:4262 , piece_id=BOWIXAXPX44RV6EW4QEFCKHFAJRVJTRDD7LCMZP7FVOLZMACPQSA, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.656357471 (3h31m46.695930601s) [noise=true , address=45.91.116.5:4299 , piece_id=YDIYWVWY26Z4ANFNY4PRG5O6M7ZT2L77NFO624BOSEXYU2AQW2BQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.656738440 (3h31m46.69631156s) [noise=true , address=45.91.116.11:4118 , piece_id=6KRA6HC46RJWHCJHODXEEHZNAF5Z6UPR7FDUXEX5QARZJ7XBTUAQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.656995702 (3h31m46.696568822s) [noise=true , address=45.91.116.12:4227 , piece_id=PXWJNK6ZBJB6RVD3MIH3DMEAXITKMYOULXIQUZZXOI4MDZ3EDJFQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.657107717 (3h31m46.696680847s) [noise=true , address=45.91.116.6:4973 , piece_id=VA6IRRQTWAPFCSQLJKVOSWLSBPLLV776ZECBW3WEKCEMX7HAVJUQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.657423230 (3h31m46.69699635s) [noise=true , address=45.91.116.7:4009 , piece_id=FBL5U3YALUTVGTOW5TGTCR2CZB5ETZQVNYVMF2YUSXHEJJ62WYIQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.657472284 (3h31m46.697045404s) [noise=true , address=45.91.116.8:4048 , piece_id=EZ55Y7EQPK3NI4TI5BSFKSOTWK5JCPTODI7QBVOHIZXEHIOGQGBQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.657732862 (3h31m46.697305982s) [noise=true , address=45.91.116.6:4974 , piece_id=3XY6VXEDWJHKEDMABLUQPT7QLRCCAVUXBWP5NUY6TYK62D3Q4F4Q, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.657782457 (3h31m46.697355577s) [noise=true , address=45.91.116.14:4190 , piece_id=NAOKDVREYPW3CJ2V6MGCONKP5KBT4B5OYFHTPYTPLOJ2TZ4KHXLQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.658078374 (3h31m46.697651494s) [noise=true , address=45.91.116.9:4263 , piece_id=EQEHK6BNFSLTW4CDLQ7SLZSGDZAKQ2BAENQBS5Z7PZYAK6KHA37A, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.658093553 (3h31m46.697666673s) [noise=true , address=45.91.116.14:4189 , piece_id=NTPS6BT7GHIFISATSDRKCXS5X2UMX6JX7RJWA5DKIONOJR5SLBCA, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.658359441 (3h31m46.697932561s) [noise=true , address=45.91.116.5:4297 , piece_id=ILJLOG3MTVZSB6Y7O6KK3Q7USTKPUTNHI56ZROCCMRAZHT2CVDTQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.659011078 (3h31m46.698584209s) [noise=true , address=77.90.31.16:4734 , piece_id=N2J6PBZLFSBWR4NFSHKV3ED5GWAUOYNHGT4K3GLXVTEXD26BY3KA, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.660164024 (3h31m46.699737155s) [noise=true , address=45.91.116.10:4081 , piece_id=AOCNU7BWALQFOFDOCD5FUOEUJZVNY7G7UEJEDX52GMQ44UPZKREA, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.660643432 (3h31m46.700216552s) [noise=true , address=45.91.116.11:4117 , piece_id=B5IGTWIXTRWCMBBFOZSBZF2I45DVBMO7E4CFDYMBQVYYMD5WXRFQ, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:10.660725028 (3h31m46.700298148s) [noise=true , address=77.90.31.17:4771 , piece_id=5FTW5FEE6MNGPBNDUG23XLXRDT4PIRLA75RP522JL52DPV4ZNIXA, seg_pos={0 0}, upload=3074]: Uploading piece... Sep 6 17:00:12.389402996 (3h31m48.428976116s) [upload=3074]: Next returned nil segment Sep 6 17:00:12.389419336 (3h31m48.428992456s) [upload=3074]: Waiting for error group managing segments... Sep 6 17:00:12.774471415 (3h31m48.814044545s) [noise=true , address=45.91.116.5:4299 , piece_id=YDIYWVWY26Z4ANFNY4PRG5O6M7ZT2L77NFO624BOSEXYU2AQW2BQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.806312743 (3h31m48.845885873s) [noise=true , address=45.91.116.11:4117 , piece_id=B5IGTWIXTRWCMBBFOZSBZF2I45DVBMO7E4CFDYMBQVYYMD5WXRFQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.820412504 (3h31m48.859985634s) [noise=true , address=77.90.31.16:4734 , piece_id=N2J6PBZLFSBWR4NFSHKV3ED5GWAUOYNHGT4K3GLXVTEXD26BY3KA, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.897993805 (3h31m48.937566925s) [noise=true , address=45.91.116.6:4974 , piece_id=3XY6VXEDWJHKEDMABLUQPT7QLRCCAVUXBWP5NUY6TYK62D3Q4F4Q, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.900868406 (3h31m48.940441526s) [noise=true , address=77.90.31.17:4771 , piece_id=5FTW5FEE6MNGPBNDUG23XLXRDT4PIRLA75RP522JL52DPV4ZNIXA, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.908303423 (3h31m48.947876553s) [noise=true , address=45.91.116.14:4190 , piece_id=NAOKDVREYPW3CJ2V6MGCONKP5KBT4B5OYFHTPYTPLOJ2TZ4KHXLQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.908863635 (3h31m48.948436755s) [noise=true , address=45.91.116.12:4227 , piece_id=PXWJNK6ZBJB6RVD3MIH3DMEAXITKMYOULXIQUZZXOI4MDZ3EDJFQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.928649726 (3h31m48.968222846s) [noise=true , address=45.91.116.9:4263 , piece_id=EQEHK6BNFSLTW4CDLQ7SLZSGDZAKQ2BAENQBS5Z7PZYAK6KHA37A, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:12.990396268 (3h31m49.029969388s) [noise=true , address=45.91.116.5:4297 , piece_id=ILJLOG3MTVZSB6Y7O6KK3Q7USTKPUTNHI56ZROCCMRAZHT2CVDTQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:13.097332763 (3h31m49.136905883s) [noise=true , address=45.91.116.6:4973 , piece_id=VA6IRRQTWAPFCSQLJKVOSWLSBPLLV776ZECBW3WEKCEMX7HAVJUQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:13.285622427 (3h31m49.325195547s) [noise=true , address=45.91.116.10:4081 , piece_id=AOCNU7BWALQFOFDOCD5FUOEUJZVNY7G7UEJEDX52GMQ44UPZKREA, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:13.415767089 (3h31m49.455340209s) [noise=true , address=45.91.116.14:4189 , piece_id=NTPS6BT7GHIFISATSDRKCXS5X2UMX6JX7RJWA5DKIONOJR5SLBCA, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:13.529442608 (3h31m49.569015728s) [noise=true , address=45.91.116.8:4048 , piece_id=EZ55Y7EQPK3NI4TI5BSFKSOTWK5JCPTODI7QBVOHIZXEHIOGQGBQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: <nil> Sep 6 17:00:13.529481111 (3h31m49.569054231s) [seg_pos={0 0}, upload=3074]: Segment reached optimal threshold of 13 pieces. Sep 6 17:00:13.529729416 (3h31m49.569302537s) [noise=true , address=45.91.116.7:4009 , piece_id=FBL5U3YALUTVGTOW5TGTCR2CZB5ETZQVNYVMF2YUSXHEJJ62WYIQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:42532->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:42532->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42532->45.91.116.7:4009: use of closed network connection Sep 6 17:00:13.529784843 (3h31m49.569357963s) [noise=true , address=45.91.116.11:4118 , piece_id=6KRA6HC46RJWHCJHODXEEHZNAF5Z6UPR7FDUXEX5QARZJ7XBTUAQ, seg_pos={0 0}, upload=3074]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 6 17:00:13.529833165 (3h31m49.569406285s) [noise=true , address=45.91.116.9:4262 , piece_id=BOWIXAXPX44RV6EW4QEFCKHFAJRVJTRDD7LCMZP7FVOLZMACPQSA, seg_pos={0 0}, upload=3074]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Sep 6 17:00:13.529855438 (3h31m49.569428558s) [seg_pos={0 0}, upload=3074]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 17:00:13.529928617 (3h31m49.569501737s) [upload=3074]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 17:00:13.529932845 (3h31m49.569505965s) [upload=3074]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 17:00:13.855937868 (3h31m49.895510988s) [upload=3074]: Done uploading segments... Sep 6 17:00:13.855956062 (3h31m49.895529192s) [upload=3074]: Upload finished. err: <nil> Duration: 30.002530315s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [X..............] duration: 632.555288ms remote: "tcp:13.233.175.166:443" response headers: X-Cluster-Name=ap-south-1-prod-hosting-red Date=Sat, 06 Sep 2025 16:49:03 GMT Content-Type=text/html Content-Length=164 target: https://www.storj.io/dcs-satellites FAIL (2025-09-06 16:59:55 UTC) orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n" Duration: 632.578653ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.X.............] duration: 648.588469ms remote: "tcp:13.233.175.166:443" response headers: X-Cluster-Name=ap-south-1-prod-hosting-red Date=Sat, 06 Sep 2025 16:49:03 GMT Content-Type=text/html Content-Length=164 target: https://www.storj.io/dcs-satellites FAIL (2025-09-06 16:59:22 UTC) orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n" Duration: 648.611001ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [XX..X..X...XX..XX..X.X.XX....XX.XXX.....X..X....X......X.XX.] duration: 9.198595147s FAIL (2025-09-06 16:59:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 16:58:21.243205021 (3h29m57.282778141s) [upload=3048]: Starting upload Sep 6 16:58:21.243291957 (3h29m57.282865077s) [upload=3048]: Done starting upload Sep 6 16:58:21.243440221 (3h29m57.283013341s) [upload=3048]: Uploading segments... Sep 6 16:58:21.243645924 (3h29m57.283219054s) [upload=3048]: Got next segment. Inline: false Sep 6 16:58:21.243693826 (3h29m57.283266956s) [upload=3048]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 16:58:21.243699447 (3h29m57.283272577s) [upload=3048]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 16:58:21.728276799 (3h29m57.767849969s) [seg_pos={0 0}, upload=3048]: Begin upload segment... Sep 6 16:58:21.728341903 (3h29m57.767915083s) [seg_pos={0 0}, upload=3048]: Done begin upload segment. Sep 6 16:58:21.728376489 (3h29m57.767949609s) [noise=true , address=45.91.116.7:4010 , piece_id=WYGXS5FVWSL6EPFF2OWQFPZNANCP4WP5CC5RCHRTABHQK5N5WJ5Q, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.728383353 (3h29m57.767956473s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUHEC6DY5K5IASFHH3ZEQ73MA4XAAV5PTF36D6H5IV4SBG24LSQ, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.729237898 (3h29m57.768811018s) [noise=true , address=45.91.116.11:4118 , piece_id=2GDTVKJMUT3GMXEDT5OJB6PGPAWTMLBOIRGE5CXYTZOQ7NSY6M6A, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.729563080 (3h29m57.76913621s) [noise=true , address=45.91.116.8:4048 , piece_id=TBH32I77EJKOKFQM4FIZLRJ3OXC47ZZBXSTKHKU7WRS6CY5IBEYQ, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.730195610 (3h29m57.76976873s) [noise=true , address=45.91.116.10:4083 , piece_id=OCFGDLFK3PVLHGZU65R3KJYGPAP3JFJH2ICH3R6OOL24K4TKGTHQ, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.730651453 (3h29m57.770224573s) [noise=true , address=45.91.116.12:4226 , piece_id=SAO5F273VKA7NSILFBHBOSBXE3YURDOADTZYEP65XDYLG5EXQEHA, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.730993387 (3h29m57.770566507s) [noise=true , address=77.90.31.17:4771 , piece_id=7NK6RJBDAZCXNDBAL4G6FHOI2LV6MFTJBW2ZEHZDTQBO6YWDFZSA, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.731057228 (3h29m57.770630359s) [noise=true , address=45.91.116.13:4153 , piece_id=YQW7CS4R7GLGG7PYOVNKX72OB56AOKRFZGTNGJS6B5HGCSSVKLOQ, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.731581272 (3h29m57.771154392s) [noise=true , address=45.91.116.5:4299 , piece_id=V5QG25DXISWWLHPKK5LFES57NDGT62RFBNAWPR3DP2HASDM7VSMQ, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.732003079 (3h29m57.771576199s) [noise=true , address=45.91.116.9:4261 , piece_id=6DVXFXFKI3ZIMWD6PZALRAH666R5SLOFDOOSQ4OZA5YU2ETV7T2Q, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.732160791 (3h29m57.771733911s) [noise=true , address=45.91.116.9:4262 , piece_id=RDDPNGNEU5UB2UV76IJOZWBKZEPB33X4Y4SPRDZ3C4VEXVEYNNGQ, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.732482707 (3h29m57.772055827s) [noise=true , address=45.91.116.4:4940 , piece_id=ZYZODZ3AUKKAAKVCGDILXHWG6C3VIVQ5ST6BZCAKHIGKPX2CJB6Q, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.732805885 (3h29m57.772379005s) [noise=true , address=45.91.116.11:4117 , piece_id=QESZFGC7PLK6PAQZXETGJIF6A6RSJ5JU6AKHMFC6DOZMTQRHFHQQ, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.733207954 (3h29m57.772781074s) [noise=true , address=45.91.116.8:4047 , piece_id=EUMYGNIRJEO2PNNQIDZQKPM2UAPFAGIT3GTBZ7BQKRO3LZNYYNNQ, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.733680668 (3h29m57.773253789s) [noise=true , address=77.90.31.19:4841 , piece_id=3QP3UHWR5HPEUDRDYE36TT4YK27XEAJTPF3OUDNH3MAHDJLBCUNQ, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.734323589 (3h29m57.773896709s) [noise=true , address=45.91.116.12:4225 , piece_id=QSXZF6WDIAZSKN4I256MWS5OGN55XOBU6D6T6J4N2SPX7FKMBT2A, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:23.543448180 (3h29m59.58302131s) [upload=3048]: Next returned nil segment Sep 6 16:58:23.543469050 (3h29m59.58304218s) [upload=3048]: Waiting for error group managing segments... Sep 6 16:58:23.873461722 (3h29m59.913034842s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUHEC6DY5K5IASFHH3ZEQ73MA4XAAV5PTF36D6H5IV4SBG24LSQ, seg_pos={0 0}, upload=3048]: Done uploading piece. err: <nil> Sep 6 16:58:23.881183278 (3h29m59.920756398s) [noise=true , address=45.91.116.13:4153 , piece_id=YQW7CS4R7GLGG7PYOVNKX72OB56AOKRFZGTNGJS6B5HGCSSVKLOQ, seg_pos={0 0}, upload=3048]: Done uploading piece. err: <nil> Sep 6 16:58:23.896601581 (3h29m59.936174701s) [noise=true , address=45.91.116.5:4299 , piece_id=V5QG25DXISWWLHPKK5LFES57NDGT62RFBNAWPR3DP2HASDM7VSMQ, seg_pos={0 0}, upload=3048]: Done uploading piece. err: <nil> Sep 6 16:58:23.926697729 (3h29m59.966270849s) [noise=true , address=45.91.116.8:4048 , piece_id=TBH32I77EJKOKFQM4FIZLRJ3OXC47ZZBXSTKHKU7WRS6CY5IBEYQ, seg_pos={0 0}, upload=3048]: Done uploading piece. err: <nil> Sep 6 16:58:23.941604183 (3h29m59.981177303s) [noise=true , address=45.91.116.11:4118 , piece_id=2GDTVKJMUT3GMXEDT5OJB6PGPAWTMLBOIRGE5CXYTZOQ7NSY6M6A, seg_pos={0 0}, upload=3048]: Done uploading piece. err: <nil> Sep 6 16:58:23.978869250 (3h30m0.01844238s) [noise=true , address=45.91.116.9:4261 , piece_id=6DVXFXFKI3ZIMWD6PZALRAH666R5SLOFDOOSQ4OZA5YU2ETV7T2Q, seg_pos={0 0}, upload=3048]: Done uploading piece. err: <nil> Sep 6 16:58:23.990480568 (3h30m0.030053688s) [noise=true , address=45.91.116.10:4083 , piece_id=OCFGDLFK3PVLHGZU65R3KJYGPAP3JFJH2ICH3R6OOL24K4TKGTHQ, seg_pos={0 0}, upload=3048]: Done uploading piece. err: <nil> Sep 6 16:58:23.998288028 (3h30m0.037861148s) [noise=true , address=45.91.116.12:4226 , piece_id=SAO5F273VKA7NSILFBHBOSBXE3YURDOADTZYEP65XDYLG5EXQEHA, seg_pos={0 0}, upload=3048]: Done uploading piece. err: <nil> Sep 6 16:58:24.064964022 (3h30m0.104537182s) [noise=true , address=45.91.116.4:4940 , piece_id=ZYZODZ3AUKKAAKVCGDILXHWG6C3VIVQ5ST6BZCAKHIGKPX2CJB6Q, seg_pos={0 0}, upload=3048]: Done uploading piece. err: <nil> Sep 6 16:58:24.074367326 (3h30m0.113940436s) [noise=true , address=45.91.116.8:4047 , piece_id=EUMYGNIRJEO2PNNQIDZQKPM2UAPFAGIT3GTBZ7BQKRO3LZNYYNNQ, seg_pos={0 0}, upload=3048]: Done uploading piece. err: <nil> Sep 6 16:58:24.093373173 (3h30m0.132946293s) [noise=true , address=45.91.116.12:4225 , piece_id=QSXZF6WDIAZSKN4I256MWS5OGN55XOBU6D6T6J4N2SPX7FKMBT2A, seg_pos={0 0}, upload=3048]: Done uploading piece. err: <nil> Sep 6 16:58:24.137958638 (3h30m0.177531758s) [noise=true , address=77.90.31.17:4771 , piece_id=7NK6RJBDAZCXNDBAL4G6FHOI2LV6MFTJBW2ZEHZDTQBO6YWDFZSA, seg_pos={0 0}, upload=3048]: Done uploading piece. err: <nil> Sep 6 16:58:24.170001943 (3h30m0.209575063s) [noise=true , address=45.91.116.7:4010 , piece_id=WYGXS5FVWSL6EPFF2OWQFPZNANCP4WP5CC5RCHRTABHQK5N5WJ5Q, seg_pos={0 0}, upload=3048]: Done uploading piece. err: <nil> Sep 6 16:58:24.170025038 (3h30m0.209598158s) [seg_pos={0 0}, upload=3048]: Segment reached optimal threshold of 13 pieces. Sep 6 16:58:24.170223317 (3h30m0.209796437s) [noise=true , address=45.91.116.11:4117 , piece_id=QESZFGC7PLK6PAQZXETGJIF6A6RSJ5JU6AKHMFC6DOZMTQRHFHQQ, seg_pos={0 0}, upload=3048]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 6 16:58:24.170234127 (3h30m0.209807237s) [noise=true , address=45.91.116.9:4262 , piece_id=RDDPNGNEU5UB2UV76IJOZWBKZEPB33X4Y4SPRDZ3C4VEXVEYNNGQ, seg_pos={0 0}, upload=3048]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Sep 6 16:58:24.170336353 (3h30m0.209909473s) [noise=true , address=77.90.31.19:4841 , piece_id=3QP3UHWR5HPEUDRDYE36TT4YK27XEAJTPF3OUDNH3MAHDJLBCUNQ, seg_pos={0 0}, upload=3048]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:42194->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:42194->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42194->77.90.31.19:4841: use of closed network connection Sep 6 16:58:24.170356261 (3h30m0.209929381s) [seg_pos={0 0}, upload=3048]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 16:58:24.170465210 (3h30m0.21003833s) [upload=3048]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 16:58:24.170469508 (3h30m0.210042618s) [upload=3048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 16:58:24.569679759 (3h30m0.609252879s) [upload=3048]: Done uploading segments... Sep 6 16:58:24.569701540 (3h30m0.60927466s) [upload=3048]: Upload finished. err: <nil> Duration: 9.198614494s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..X............] duration: 1.124964396s remote: "tcp:13.233.175.166:443" response headers: X-Cluster-Name=ap-south-1-prod-hosting-red Date=Sat, 06 Sep 2025 16:49:03 GMT Content-Type=text/html Content-Length=164 target: https://www.storj.io/dcs-satellites FAIL (2025-09-06 16:58:51 UTC) orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n" Duration: 1.125053525s core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [.XX..X..X...XX..XX..X.X.XX....XX.XXX.....X..X....X......X.XX] duration: 30.002981973s FAIL (2025-09-06 16:58:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 16:58:21.243205021 (3h29m57.282778141s) [upload=3048]: Starting upload Sep 6 16:58:21.243291957 (3h29m57.282865077s) [upload=3048]: Done starting upload Sep 6 16:58:21.243440221 (3h29m57.283013341s) [upload=3048]: Uploading segments... Sep 6 16:58:21.243645924 (3h29m57.283219054s) [upload=3048]: Got next segment. Inline: false Sep 6 16:58:21.243693826 (3h29m57.283266956s) [upload=3048]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 16:58:21.243699447 (3h29m57.283272577s) [upload=3048]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 16:58:21.728276799 (3h29m57.767849969s) [seg_pos={0 0}, upload=3048]: Begin upload segment... Sep 6 16:58:21.728341903 (3h29m57.767915083s) [seg_pos={0 0}, upload=3048]: Done begin upload segment. Sep 6 16:58:21.728376489 (3h29m57.767949609s) [noise=true , address=45.91.116.7:4010 , piece_id=WYGXS5FVWSL6EPFF2OWQFPZNANCP4WP5CC5RCHRTABHQK5N5WJ5Q, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.728383353 (3h29m57.767956473s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUHEC6DY5K5IASFHH3ZEQ73MA4XAAV5PTF36D6H5IV4SBG24LSQ, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.729237898 (3h29m57.768811018s) [noise=true , address=45.91.116.11:4118 , piece_id=2GDTVKJMUT3GMXEDT5OJB6PGPAWTMLBOIRGE5CXYTZOQ7NSY6M6A, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.729563080 (3h29m57.76913621s) [noise=true , address=45.91.116.8:4048 , piece_id=TBH32I77EJKOKFQM4FIZLRJ3OXC47ZZBXSTKHKU7WRS6CY5IBEYQ, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.730195610 (3h29m57.76976873s) [noise=true , address=45.91.116.10:4083 , piece_id=OCFGDLFK3PVLHGZU65R3KJYGPAP3JFJH2ICH3R6OOL24K4TKGTHQ, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.730651453 (3h29m57.770224573s) [noise=true , address=45.91.116.12:4226 , piece_id=SAO5F273VKA7NSILFBHBOSBXE3YURDOADTZYEP65XDYLG5EXQEHA, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.730993387 (3h29m57.770566507s) [noise=true , address=77.90.31.17:4771 , piece_id=7NK6RJBDAZCXNDBAL4G6FHOI2LV6MFTJBW2ZEHZDTQBO6YWDFZSA, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.731057228 (3h29m57.770630359s) [noise=true , address=45.91.116.13:4153 , piece_id=YQW7CS4R7GLGG7PYOVNKX72OB56AOKRFZGTNGJS6B5HGCSSVKLOQ, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.731581272 (3h29m57.771154392s) [noise=true , address=45.91.116.5:4299 , piece_id=V5QG25DXISWWLHPKK5LFES57NDGT62RFBNAWPR3DP2HASDM7VSMQ, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.732003079 (3h29m57.771576199s) [noise=true , address=45.91.116.9:4261 , piece_id=6DVXFXFKI3ZIMWD6PZALRAH666R5SLOFDOOSQ4OZA5YU2ETV7T2Q, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.732160791 (3h29m57.771733911s) [noise=true , address=45.91.116.9:4262 , piece_id=RDDPNGNEU5UB2UV76IJOZWBKZEPB33X4Y4SPRDZ3C4VEXVEYNNGQ, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.732482707 (3h29m57.772055827s) [noise=true , address=45.91.116.4:4940 , piece_id=ZYZODZ3AUKKAAKVCGDILXHWG6C3VIVQ5ST6BZCAKHIGKPX2CJB6Q, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.732805885 (3h29m57.772379005s) [noise=true , address=45.91.116.11:4117 , piece_id=QESZFGC7PLK6PAQZXETGJIF6A6RSJ5JU6AKHMFC6DOZMTQRHFHQQ, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.733207954 (3h29m57.772781074s) [noise=true , address=45.91.116.8:4047 , piece_id=EUMYGNIRJEO2PNNQIDZQKPM2UAPFAGIT3GTBZ7BQKRO3LZNYYNNQ, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.733680668 (3h29m57.773253789s) [noise=true , address=77.90.31.19:4841 , piece_id=3QP3UHWR5HPEUDRDYE36TT4YK27XEAJTPF3OUDNH3MAHDJLBCUNQ, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:21.734323589 (3h29m57.773896709s) [noise=true , address=45.91.116.12:4225 , piece_id=QSXZF6WDIAZSKN4I256MWS5OGN55XOBU6D6T6J4N2SPX7FKMBT2A, seg_pos={0 0}, upload=3048]: Uploading piece... Sep 6 16:58:23.543448180 (3h29m59.58302131s) [upload=3048]: Next returned nil segment Sep 6 16:58:23.543469050 (3h29m59.58304218s) [upload=3048]: Waiting for error group managing segments... Sep 6 16:58:23.873461722 (3h29m59.913034842s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUHEC6DY5K5IASFHH3ZEQ73MA4XAAV5PTF36D6H5IV4SBG24LSQ, seg_pos={0 0}, upload=3048]: Done uploading piece. err: <nil> Sep 6 16:58:23.881183278 (3h29m59.920756398s) [noise=true , address=45.91.116.13:4153 , piece_id=YQW7CS4R7GLGG7PYOVNKX72OB56AOKRFZGTNGJS6B5HGCSSVKLOQ, seg_pos={0 0}, upload=3048]: Done uploading piece. err: <nil> Sep 6 16:58:23.896601581 (3h29m59.936174701s) [noise=true , address=45.91.116.5:4299 , piece_id=V5QG25DXISWWLHPKK5LFES57NDGT62RFBNAWPR3DP2HASDM7VSMQ, seg_pos={0 0}, upload=3048]: Done uploading piece. err: <nil> Sep 6 16:58:23.926697729 (3h29m59.966270849s) [noise=true , address=45.91.116.8:4048 , piece_id=TBH32I77EJKOKFQM4FIZLRJ3OXC47ZZBXSTKHKU7WRS6CY5IBEYQ, seg_pos={0 0}, upload=3048]: Done uploading piece. err: <nil> Sep 6 16:58:23.941604183 (3h29m59.981177303s) [noise=true , address=45.91.116.11:4118 , piece_id=2GDTVKJMUT3GMXEDT5OJB6PGPAWTMLBOIRGE5CXYTZOQ7NSY6M6A, seg_pos={0 0}, upload=3048]: Done uploading piece. err: <nil> Sep 6 16:58:23.978869250 (3h30m0.01844238s) [noise=true , address=45.91.116.9:4261 , piece_id=6DVXFXFKI3ZIMWD6PZALRAH666R5SLOFDOOSQ4OZA5YU2ETV7T2Q, seg_pos={0 0}, upload=3048]: Done uploading piece. err: <nil> Sep 6 16:58:23.990480568 (3h30m0.030053688s) [noise=true , address=45.91.116.10:4083 , piece_id=OCFGDLFK3PVLHGZU65R3KJYGPAP3JFJH2ICH3R6OOL24K4TKGTHQ, seg_pos={0 0}, upload=3048]: Done uploading piece. err: <nil> Sep 6 16:58:23.998288028 (3h30m0.037861148s) [noise=true , address=45.91.116.12:4226 , piece_id=SAO5F273VKA7NSILFBHBOSBXE3YURDOADTZYEP65XDYLG5EXQEHA, seg_pos={0 0}, upload=3048]: Done uploading piece. err: <nil> Sep 6 16:58:24.064964022 (3h30m0.104537182s) [noise=true , address=45.91.116.4:4940 , piece_id=ZYZODZ3AUKKAAKVCGDILXHWG6C3VIVQ5ST6BZCAKHIGKPX2CJB6Q, seg_pos={0 0}, upload=3048]: Done uploading piece. err: <nil> Sep 6 16:58:24.074367326 (3h30m0.113940436s) [noise=true , address=45.91.116.8:4047 , piece_id=EUMYGNIRJEO2PNNQIDZQKPM2UAPFAGIT3GTBZ7BQKRO3LZNYYNNQ, seg_pos={0 0}, upload=3048]: Done uploading piece. err: <nil> Sep 6 16:58:24.093373173 (3h30m0.132946293s) [noise=true , address=45.91.116.12:4225 , piece_id=QSXZF6WDIAZSKN4I256MWS5OGN55XOBU6D6T6J4N2SPX7FKMBT2A, seg_pos={0 0}, upload=3048]: Done uploading piece. err: <nil> Sep 6 16:58:24.137958638 (3h30m0.177531758s) [noise=true , address=77.90.31.17:4771 , piece_id=7NK6RJBDAZCXNDBAL4G6FHOI2LV6MFTJBW2ZEHZDTQBO6YWDFZSA, seg_pos={0 0}, upload=3048]: Done uploading piece. err: <nil> Sep 6 16:58:24.170001943 (3h30m0.209575063s) [noise=true , address=45.91.116.7:4010 , piece_id=WYGXS5FVWSL6EPFF2OWQFPZNANCP4WP5CC5RCHRTABHQK5N5WJ5Q, seg_pos={0 0}, upload=3048]: Done uploading piece. err: <nil> Sep 6 16:58:24.170025038 (3h30m0.209598158s) [seg_pos={0 0}, upload=3048]: Segment reached optimal threshold of 13 pieces. Sep 6 16:58:24.170223317 (3h30m0.209796437s) [noise=true , address=45.91.116.11:4117 , piece_id=QESZFGC7PLK6PAQZXETGJIF6A6RSJ5JU6AKHMFC6DOZMTQRHFHQQ, seg_pos={0 0}, upload=3048]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6): protocol: expected piece hash; context canceled; EOF Sep 6 16:58:24.170234127 (3h30m0.209807237s) [noise=true , address=45.91.116.9:4262 , piece_id=RDDPNGNEU5UB2UV76IJOZWBKZEPB33X4Y4SPRDZ3C4VEXVEYNNGQ, seg_pos={0 0}, upload=3048]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Sep 6 16:58:24.170336353 (3h30m0.209909473s) [noise=true , address=77.90.31.19:4841 , piece_id=3QP3UHWR5HPEUDRDYE36TT4YK27XEAJTPF3OUDNH3MAHDJLBCUNQ, seg_pos={0 0}, upload=3048]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:42194->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:42194->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42194->77.90.31.19:4841: use of closed network connection Sep 6 16:58:24.170356261 (3h30m0.209929381s) [seg_pos={0 0}, upload=3048]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 16:58:24.170465210 (3h30m0.21003833s) [upload=3048]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 16:58:24.170469508 (3h30m0.210042618s) [upload=3048]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 16:58:24.569679759 (3h30m0.609252879s) [upload=3048]: Done uploading segments... Sep 6 16:58:24.569701540 (3h30m0.60927466s) [upload=3048]: Upload finished. err: <nil> Duration: 30.002995057s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [X.XX..X..X...XX..XX..X.X.XX....XX.XXX.....X..X....X......X.X] duration: 30.002416231s FAIL (2025-09-06 16:58:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 16:57:46.506779639 (3h29m22.546352759s) [upload=3036]: Starting upload Sep 6 16:57:46.506840476 (3h29m22.546413607s) [upload=3036]: Done starting upload Sep 6 16:57:46.506933815 (3h29m22.546506925s) [upload=3036]: Uploading segments... Sep 6 16:57:46.507019929 (3h29m22.546593049s) [upload=3036]: Got next segment. Inline: false Sep 6 16:57:46.507066899 (3h29m22.546640019s) [upload=3036]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 16:57:46.507071779 (3h29m22.546644899s) [upload=3036]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 16:57:46.978914853 (3h29m23.018488014s) [seg_pos={0 0}, upload=3036]: Begin upload segment... Sep 6 16:57:46.978983344 (3h29m23.018556514s) [seg_pos={0 0}, upload=3036]: Done begin upload segment. Sep 6 16:57:46.979026597 (3h29m23.018599747s) [noise=true , address=45.91.116.9:4262 , piece_id=TA7KJ2D3CURFKNI5THA4DCGETLJTUO2K3QHPRI7TCJ2MCBWXKE2Q, seg_pos={0 0}, upload=3036]: Uploading piece... Sep 6 16:57:46.979056444 (3h29m23.018629564s) [noise=true , address=45.91.116.9:4261 , piece_id=WIOHV62UI476CY2JWUXKURMSHRE22GNIV27XBUEM6PWFJDFFWTUA, seg_pos={0 0}, upload=3036]: Uploading piece... Sep 6 16:57:46.979583703 (3h29m23.019156823s) [noise=true , address=45.91.116.10:4082 , piece_id=VBDTYD5LHOC7AK5KF7JBIWT3FKQSOA2KG5UZ2GYGF36ZCSHZGVVQ, seg_pos={0 0}, upload=3036]: Uploading piece... Sep 6 16:57:46.979892785 (3h29m23.019465925s) [noise=true , address=45.91.116.6:4973 , piece_id=N3HWJWOHV2SJBIYYV7JGVC2UV73XZDBOK7Q7NJXXLZALCUN7IHKQ, seg_pos={0 0}, upload=3036]: Uploading piece... Sep 6 16:57:46.979920057 (3h29m23.019493177s) [noise=true , address=45.91.116.8:4047 , piece_id=2QBAN2WAHTYC2KJQWVN5QMB2MFBLUDK3JFVQMGHXIAV7QPRKLXIA, seg_pos={0 0}, upload=3036]: Uploading piece... Sep 6 16:57:46.979979791 (3h29m23.019552901s) [noise=true , address=45.91.116.7:4009 , piece_id=OXPUJYKBBE7WZMGFGLH55MQVGM6I6IXQKBJDGLLF47XZNX7K5ZAA, seg_pos={0 0}, upload=3036]: Uploading piece... Sep 6 16:57:46.980297318 (3h29m23.019870438s) [noise=true , address=77.90.31.10:4519 , piece_id=LGVSR6USFQVOS5MP2PZ3R7SUV4JX52CMMXHRYWZL3RSR6UYYMB7A, seg_pos={0 0}, upload=3036]: Uploading piece... Sep 6 16:57:46.980603395 (3h29m23.020176545s) [noise=true , address=45.91.116.12:4227 , piece_id=MAMZSL7WILKATAKVA4XI6T5POUIKQ2LSKJTARB224N6RYF4FIPCA, seg_pos={0 0}, upload=3036]: Uploading piece... Sep 6 16:57:46.980620026 (3h29m23.020193136s) [noise=true , address=45.91.116.11:4118 , piece_id=4AOQGSIB3HQTYT66PTGLZFTANBBLTINKGOHA7SZJF554S5LCL6EA, seg_pos={0 0}, upload=3036]: Uploading piece... Sep 6 16:57:46.981720762 (3h29m23.021293882s) [noise=true , address=77.90.31.10:4520 , piece_id=63ZASJ6GA3EXW7YIGZRVCUBGVM2S7NQYON4EYCFWX7I2X3XULYNQ, seg_pos={0 0}, upload=3036]: Uploading piece... Sep 6 16:57:46.982087184 (3h29m23.021660304s) [noise=true , address=77.90.31.13:4627 , piece_id=XEJTHK6KIVOGTI7FPHIVAVEMFEXRHFZ44CJZZT4DT7WXXYJXEGQQ, seg_pos={0 0}, upload=3036]: Uploading piece... Sep 6 16:57:46.982378080 (3h29m23.0219512s) [noise=true , address=45.91.116.12:4226 , piece_id=LAX7QHSJ6ARMDUWBSYDUKVSVMJCEUESOOK525QCZE6F65N662BCQ, seg_pos={0 0}, upload=3036]: Uploading piece... Sep 6 16:57:46.982566851 (3h29m23.022139971s) [noise=true , address=77.90.31.18:4805 , piece_id=QFB2SH6MENIPQ735U2THN7HJTAHPUUPKQ27454F2FXXRB7CAMKWA, seg_pos={0 0}, upload=3036]: Uploading piece... Sep 6 16:57:46.982875272 (3h29m23.022448392s) [noise=true , address=45.91.116.10:4081 , piece_id=GWP5AOZD2WHHRTHUER3JAVTCAZHHFOSTXEZHFO3OW6C2Q2WHVGOA, seg_pos={0 0}, upload=3036]: Uploading piece... Sep 6 16:57:46.983107055 (3h29m23.022680175s) [noise=true , address=45.91.116.8:4048 , piece_id=FZPTJYMLCCDOOHTYW5PD5EQDRQIXZOWH25TW2FCAZS5JVGISICQA, seg_pos={0 0}, upload=3036]: Uploading piece... Sep 6 16:57:46.984037716 (3h29m23.023610826s) [noise=true , address=77.90.31.19:4843 , piece_id=GOPGBFTBGSEWQWUU455QIXQ6YXIJ4VUYVJIIZHWGQESKAB2BHLZA, seg_pos={0 0}, upload=3036]: Uploading piece... Sep 6 16:57:48.819545502 (3h29m24.859118622s) [upload=3036]: Next returned nil segment Sep 6 16:57:48.819562295 (3h29m24.859135415s) [upload=3036]: Waiting for error group managing segments... Sep 6 16:57:49.142270208 (3h29m25.181843328s) [noise=true , address=45.91.116.6:4973 , piece_id=N3HWJWOHV2SJBIYYV7JGVC2UV73XZDBOK7Q7NJXXLZALCUN7IHKQ, seg_pos={0 0}, upload=3036]: Done uploading piece. err: <nil> Sep 6 16:57:49.170327075 (3h29m25.209900185s) [noise=true , address=45.91.116.9:4261 , piece_id=WIOHV62UI476CY2JWUXKURMSHRE22GNIV27XBUEM6PWFJDFFWTUA, seg_pos={0 0}, upload=3036]: Done uploading piece. err: <nil> Sep 6 16:57:49.176111844 (3h29m25.215684965s) [noise=true , address=45.91.116.8:4047 , piece_id=2QBAN2WAHTYC2KJQWVN5QMB2MFBLUDK3JFVQMGHXIAV7QPRKLXIA, seg_pos={0 0}, upload=3036]: Done uploading piece. err: <nil> Sep 6 16:57:49.197847565 (3h29m25.237420685s) [noise=true , address=45.91.116.10:4082 , piece_id=VBDTYD5LHOC7AK5KF7JBIWT3FKQSOA2KG5UZ2GYGF36ZCSHZGVVQ, seg_pos={0 0}, upload=3036]: Done uploading piece. err: <nil> Sep 6 16:57:49.222359920 (3h29m25.26193304s) [noise=true , address=45.91.116.12:4226 , piece_id=LAX7QHSJ6ARMDUWBSYDUKVSVMJCEUESOOK525QCZE6F65N662BCQ, seg_pos={0 0}, upload=3036]: Done uploading piece. err: <nil> Sep 6 16:57:49.222426568 (3h29m25.261999688s) [noise=true , address=77.90.31.13:4627 , piece_id=XEJTHK6KIVOGTI7FPHIVAVEMFEXRHFZ44CJZZT4DT7WXXYJXEGQQ, seg_pos={0 0}, upload=3036]: Done uploading piece. err: <nil> Sep 6 16:57:49.261658519 (3h29m25.301231639s) [noise=true , address=45.91.116.10:4081 , piece_id=GWP5AOZD2WHHRTHUER3JAVTCAZHHFOSTXEZHFO3OW6C2Q2WHVGOA, seg_pos={0 0}, upload=3036]: Done uploading piece. err: <nil> Sep 6 16:57:49.312384443 (3h29m25.351957563s) [noise=true , address=45.91.116.9:4262 , piece_id=TA7KJ2D3CURFKNI5THA4DCGETLJTUO2K3QHPRI7TCJ2MCBWXKE2Q, seg_pos={0 0}, upload=3036]: Done uploading piece. err: <nil> Sep 6 16:57:49.326068256 (3h29m25.365641376s) [noise=true , address=77.90.31.18:4805 , piece_id=QFB2SH6MENIPQ735U2THN7HJTAHPUUPKQ27454F2FXXRB7CAMKWA, seg_pos={0 0}, upload=3036]: Done uploading piece. err: <nil> Sep 6 16:57:49.397872405 (3h29m25.437445525s) [noise=true , address=45.91.116.12:4227 , piece_id=MAMZSL7WILKATAKVA4XI6T5POUIKQ2LSKJTARB224N6RYF4FIPCA, seg_pos={0 0}, upload=3036]: Done uploading piece. err: <nil> Sep 6 16:57:49.490743905 (3h29m25.530317025s) [noise=true , address=45.91.116.7:4009 , piece_id=OXPUJYKBBE7WZMGFGLH55MQVGM6I6IXQKBJDGLLF47XZNX7K5ZAA, seg_pos={0 0}, upload=3036]: Done uploading piece. err: <nil> Sep 6 16:57:49.611244033 (3h29m25.650817153s) [noise=true , address=77.90.31.19:4843 , piece_id=GOPGBFTBGSEWQWUU455QIXQ6YXIJ4VUYVJIIZHWGQESKAB2BHLZA, seg_pos={0 0}, upload=3036]: Done uploading piece. err: <nil> Sep 6 16:57:49.991964644 (3h29m26.031537764s) [noise=true , address=45.91.116.11:4118 , piece_id=4AOQGSIB3HQTYT66PTGLZFTANBBLTINKGOHA7SZJF554S5LCL6EA, seg_pos={0 0}, upload=3036]: Done uploading piece. err: <nil> Sep 6 16:57:49.991991345 (3h29m26.031564465s) [seg_pos={0 0}, upload=3036]: Segment reached optimal threshold of 13 pieces. Sep 6 16:57:49.992157793 (3h29m26.031730913s) [noise=true , address=77.90.31.10:4519 , piece_id=LGVSR6USFQVOS5MP2PZ3R7SUV4JX52CMMXHRYWZL3RSR6UYYMB7A, seg_pos={0 0}, upload=3036]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Sep 6 16:57:49.992165668 (3h29m26.031738788s) [noise=true , address=45.91.116.8:4048 , piece_id=FZPTJYMLCCDOOHTYW5PD5EQDRQIXZOWH25TW2FCAZS5JVGISICQA, seg_pos={0 0}, upload=3036]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 6 16:57:49.992231134 (3h29m26.031804254s) [noise=true , address=77.90.31.10:4520 , piece_id=63ZASJ6GA3EXW7YIGZRVCUBGVM2S7NQYON4EYCFWX7I2X3XULYNQ, seg_pos={0 0}, upload=3036]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:56046->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:56046->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56046->77.90.31.10:4520: use of closed network connection Sep 6 16:57:49.992280258 (3h29m26.031853378s) [seg_pos={0 0}, upload=3036]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 16:57:49.992338099 (3h29m26.031911219s) [upload=3036]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 16:57:49.992341595 (3h29m26.031914705s) [upload=3036]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 16:57:50.459760033 (3h29m26.499333154s) [upload=3036]: Done uploading segments... Sep 6 16:57:50.459775363 (3h29m26.499348473s) [upload=3036]: Upload finished. err: <nil> Duration: 30.002439607s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [...X...........] duration: 924.598887ms remote: "tcp:13.233.175.166:443" response headers: X-Cluster-Name=ap-south-1-prod-hosting-red Date=Sat, 06 Sep 2025 16:49:03 GMT Content-Type=text/html Content-Length=164 target: https://www.storj.io/dcs-satellites FAIL (2025-09-06 16:57:50 UTC) orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n" Duration: 924.618574ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [XX.XX..X..X...XX..XX..X.X.XX....XX.XXX.....X..X....X......X.] duration: 29.849342631s FAIL (2025-09-06 16:57:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 16:56:33.380099438 (3h28m9.419672558s) [upload=3020]: Starting upload Sep 6 16:56:33.380194350 (3h28m9.41976747s) [upload=3020]: Done starting upload Sep 6 16:56:33.380405525 (3h28m9.419978645s) [upload=3020]: Uploading segments... Sep 6 16:56:33.380508591 (3h28m9.420081701s) [upload=3020]: Got next segment. Inline: false Sep 6 16:56:33.380556363 (3h28m9.420129483s) [upload=3020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 16:56:33.380563907 (3h28m9.420137027s) [upload=3020]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 16:56:33.650960813 (3h28m9.690533933s) [seg_pos={0 0}, upload=3020]: Begin upload segment... Sep 6 16:56:33.651020087 (3h28m9.690593227s) [seg_pos={0 0}, upload=3020]: Done begin upload segment. Sep 6 16:56:33.651031598 (3h28m9.690604718s) [noise=true , address=45.91.116.12:4225 , piece_id=IVQZNOHCINPMDXW6MSQA5HBWPYSNIGWCTLRNRV374M5OCNBBNM4A, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.651045245 (3h28m9.690618405s) [noise=true , address=45.91.116.5:4299 , piece_id=UGYUAYFSFZN472CFWWGB5ITJLPHOZS7X6B3DV7MPN2CHE5KN4GAQ, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.651400505 (3h28m9.690973625s) [noise=true , address=45.91.116.5:4297 , piece_id=WM7LU5DMM5TIAOYGE6VWCLIQ7QCGLDUTA6FH3DWEEW34GI62CTDQ, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.651712030 (3h28m9.69128515s) [noise=true , address=45.91.116.8:4045 , piece_id=KMC7WJE6DAYEK5X6R2ZGGZVBARZE5N6ZZGSQPBCQQB5UWBQI3XHA, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.651959444 (3h28m9.691532564s) [noise=true , address=77.90.31.15:4698 , piece_id=QUFN2FMBZ7LEHUG4KGFJAK2SH3AXC5IWZXPJWVU5J4ZIAMKXBG4A, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.652208931 (3h28m9.691782051s) [noise=true , address=45.91.116.7:4009 , piece_id=QUXR35LPTYERLQFFRSPDFQJSL4ZLG3UC24TLAO3KM7GWIIXVMH5A, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.652447387 (3h28m9.692020508s) [noise=true , address=45.91.116.14:4190 , piece_id=YUS2SSQ3X4EJWB4C2HMULCEI4HQXOZYK72PDPKTQEZLRQZVIRIGA, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.652731360 (3h28m9.692304481s) [noise=true , address=77.90.31.16:4734 , piece_id=TDZ5HN6FOC2DVU3MQEXWEK6DMI2CEWDBFQFXUWLSCK4JD2JXXBJA, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.653055210 (3h28m9.69262833s) [noise=true , address=45.91.116.6:4974 , piece_id=5IU6JHBQ55BM2CF55RM3CQAAHIUJC7RKFTQOCEWAOCBRHUXGE3RA, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.653382968 (3h28m9.692956078s) [noise=true , address=77.90.31.18:4807 , piece_id=PBRG7U66P3LKM3DD2TG5RANULZTEWWHUA6WGG6T6BGPLFNH5RD2Q, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.653599151 (3h28m9.693172271s) [noise=true , address=77.90.31.19:4841 , piece_id=2PKDYTCWZD3N6SB2XQORTLAYWZMI4QVBVB2BLJJAED3LYBNWT2AA, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.653863287 (3h28m9.693436407s) [noise=true , address=45.91.116.10:4082 , piece_id=P2TPQBFLBS3XA6ZRRURO3LM6IQNRHANQNOCSMHUG6EYGWVZLL6YQ, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.654064993 (3h28m9.693638113s) [noise=true , address=45.91.116.4:4938 , piece_id=M5ZS5QZRPBMBOSLT2KTFY7PF72NBOCG2LSTDMBV3QQL3UMLOL6MQ, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.654251269 (3h28m9.693824389s) [noise=true , address=45.91.116.13:4155 , piece_id=P4OLS7LLAU5OEV7P4NNU7L53OFRCZUROLA4HGMXPPMQWTYZB2O3Q, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.654432817 (3h28m9.694005937s) [noise=true , address=45.91.116.12:4226 , piece_id=FTFHJJ27WCDYQBKMRYQXXGFKKCXUIXO5TYZKXCGUYQIUDYDJUVGA, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.654639201 (3h28m9.694212311s) [noise=true , address=45.91.116.8:4047 , piece_id=YNL66X7DUB32UU2ZNPWLHIF4Y3HRPK3JUF4ECXAFZFQ3BTXH52KA, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:35.469006996 (3h28m11.508580116s) [upload=3020]: Next returned nil segment Sep 6 16:56:35.469024489 (3h28m11.508597609s) [upload=3020]: Waiting for error group managing segments... Sep 6 16:56:35.773140196 (3h28m11.812713316s) [noise=true , address=77.90.31.15:4698 , piece_id=QUFN2FMBZ7LEHUG4KGFJAK2SH3AXC5IWZXPJWVU5J4ZIAMKXBG4A, seg_pos={0 0}, upload=3020]: Done uploading piece. err: <nil> Sep 6 16:56:35.790480969 (3h28m11.830054089s) [noise=true , address=45.91.116.13:4155 , piece_id=P4OLS7LLAU5OEV7P4NNU7L53OFRCZUROLA4HGMXPPMQWTYZB2O3Q, seg_pos={0 0}, upload=3020]: Done uploading piece. err: <nil> Sep 6 16:56:35.802972712 (3h28m11.842545832s) [noise=true , address=45.91.116.5:4297 , piece_id=WM7LU5DMM5TIAOYGE6VWCLIQ7QCGLDUTA6FH3DWEEW34GI62CTDQ, seg_pos={0 0}, upload=3020]: Done uploading piece. err: <nil> Sep 6 16:56:35.864354275 (3h28m11.903927395s) [noise=true , address=45.91.116.7:4009 , piece_id=QUXR35LPTYERLQFFRSPDFQJSL4ZLG3UC24TLAO3KM7GWIIXVMH5A, seg_pos={0 0}, upload=3020]: Done uploading piece. err: <nil> Sep 6 16:56:35.873268152 (3h28m11.912841262s) [noise=true , address=45.91.116.5:4299 , piece_id=UGYUAYFSFZN472CFWWGB5ITJLPHOZS7X6B3DV7MPN2CHE5KN4GAQ, seg_pos={0 0}, upload=3020]: Done uploading piece. err: <nil> Sep 6 16:56:35.917053294 (3h28m11.956626414s) [noise=true , address=45.91.116.8:4047 , piece_id=YNL66X7DUB32UU2ZNPWLHIF4Y3HRPK3JUF4ECXAFZFQ3BTXH52KA, seg_pos={0 0}, upload=3020]: Done uploading piece. err: <nil> Sep 6 16:56:35.927307436 (3h28m11.966880556s) [noise=true , address=45.91.116.10:4082 , piece_id=P2TPQBFLBS3XA6ZRRURO3LM6IQNRHANQNOCSMHUG6EYGWVZLL6YQ, seg_pos={0 0}, upload=3020]: Done uploading piece. err: <nil> Sep 6 16:56:35.931391341 (3h28m11.970964461s) [noise=true , address=77.90.31.16:4734 , piece_id=TDZ5HN6FOC2DVU3MQEXWEK6DMI2CEWDBFQFXUWLSCK4JD2JXXBJA, seg_pos={0 0}, upload=3020]: Done uploading piece. err: <nil> Sep 6 16:56:35.931467917 (3h28m11.971041037s) [noise=true , address=45.91.116.12:4225 , piece_id=IVQZNOHCINPMDXW6MSQA5HBWPYSNIGWCTLRNRV374M5OCNBBNM4A, seg_pos={0 0}, upload=3020]: Done uploading piece. err: <nil> Sep 6 16:56:35.978933703 (3h28m12.018506824s) [noise=true , address=45.91.116.6:4974 , piece_id=5IU6JHBQ55BM2CF55RM3CQAAHIUJC7RKFTQOCEWAOCBRHUXGE3RA, seg_pos={0 0}, upload=3020]: Done uploading piece. err: <nil> Sep 6 16:56:36.017688511 (3h28m12.057261631s) [noise=true , address=45.91.116.4:4938 , piece_id=M5ZS5QZRPBMBOSLT2KTFY7PF72NBOCG2LSTDMBV3QQL3UMLOL6MQ, seg_pos={0 0}, upload=3020]: Done uploading piece. err: <nil> Sep 6 16:56:36.038924416 (3h28m12.078497536s) [noise=true , address=45.91.116.14:4190 , piece_id=YUS2SSQ3X4EJWB4C2HMULCEI4HQXOZYK72PDPKTQEZLRQZVIRIGA, seg_pos={0 0}, upload=3020]: Done uploading piece. err: <nil> Sep 6 16:56:36.042317719 (3h28m12.081890839s) [noise=true , address=45.91.116.8:4045 , piece_id=KMC7WJE6DAYEK5X6R2ZGGZVBARZE5N6ZZGSQPBCQQB5UWBQI3XHA, seg_pos={0 0}, upload=3020]: Done uploading piece. err: <nil> Sep 6 16:56:36.042329773 (3h28m12.081902883s) [seg_pos={0 0}, upload=3020]: Segment reached optimal threshold of 13 pieces. Sep 6 16:56:36.042478116 (3h28m12.082051226s) [noise=true , address=77.90.31.18:4807 , piece_id=PBRG7U66P3LKM3DD2TG5RANULZTEWWHUA6WGG6T6BGPLFNH5RD2Q, seg_pos={0 0}, upload=3020]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS): protocol: expected piece hash; context canceled; EOF Sep 6 16:56:36.042624676 (3h28m12.082197796s) [noise=true , address=77.90.31.19:4841 , piece_id=2PKDYTCWZD3N6SB2XQORTLAYWZMI4QVBVB2BLJJAED3LYBNWT2AA, seg_pos={0 0}, upload=3020]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:35860->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:35860->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35860->77.90.31.19:4841: use of closed network connection Sep 6 16:56:36.042639154 (3h28m12.082212274s) [noise=true , address=45.91.116.12:4226 , piece_id=FTFHJJ27WCDYQBKMRYQXXGFKKCXUIXO5TYZKXCGUYQIUDYDJUVGA, seg_pos={0 0}, upload=3020]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: write tcp 188.166.225.170:39776->45.91.116.12:4226: use of closed network connection; write tcp 188.166.225.170:39776->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39776->45.91.116.12:4226: use of closed network connection Sep 6 16:56:36.042659133 (3h28m12.082232253s) [seg_pos={0 0}, upload=3020]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 16:56:36.042722323 (3h28m12.082295444s) [upload=3020]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 16:56:36.042726211 (3h28m12.082299331s) [upload=3020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 16:56:36.355029354 (3h28m12.394602474s) [upload=3020]: Done uploading segments... Sep 6 16:56:36.355046867 (3h28m12.394619977s) [upload=3020]: Upload finished. err: <nil> Duration: 29.84936879s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [....X..........] duration: 632.004814ms remote: "tcp:13.233.175.166:443" response headers: X-Cluster-Name=ap-south-1-prod-hosting-red Date=Sat, 06 Sep 2025 16:49:03 GMT Content-Type=text/html Content-Length=164 target: https://www.storj.io/dcs-satellites FAIL (2025-09-06 16:57:12 UTC) orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n" Duration: 632.026737ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 24 out of 60 is failed (threshold=3) [XXX.XX..X..X...XX..XX..X.X.XX....XX.XXX.....X..X....X......X] duration: 30.001239163s FAIL (2025-09-06 16:57:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 16:56:33.380099438 (3h28m9.419672558s) [upload=3020]: Starting upload Sep 6 16:56:33.380194350 (3h28m9.41976747s) [upload=3020]: Done starting upload Sep 6 16:56:33.380405525 (3h28m9.419978645s) [upload=3020]: Uploading segments... Sep 6 16:56:33.380508591 (3h28m9.420081701s) [upload=3020]: Got next segment. Inline: false Sep 6 16:56:33.380556363 (3h28m9.420129483s) [upload=3020]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 16:56:33.380563907 (3h28m9.420137027s) [upload=3020]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 16:56:33.650960813 (3h28m9.690533933s) [seg_pos={0 0}, upload=3020]: Begin upload segment... Sep 6 16:56:33.651020087 (3h28m9.690593227s) [seg_pos={0 0}, upload=3020]: Done begin upload segment. Sep 6 16:56:33.651031598 (3h28m9.690604718s) [noise=true , address=45.91.116.12:4225 , piece_id=IVQZNOHCINPMDXW6MSQA5HBWPYSNIGWCTLRNRV374M5OCNBBNM4A, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.651045245 (3h28m9.690618405s) [noise=true , address=45.91.116.5:4299 , piece_id=UGYUAYFSFZN472CFWWGB5ITJLPHOZS7X6B3DV7MPN2CHE5KN4GAQ, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.651400505 (3h28m9.690973625s) [noise=true , address=45.91.116.5:4297 , piece_id=WM7LU5DMM5TIAOYGE6VWCLIQ7QCGLDUTA6FH3DWEEW34GI62CTDQ, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.651712030 (3h28m9.69128515s) [noise=true , address=45.91.116.8:4045 , piece_id=KMC7WJE6DAYEK5X6R2ZGGZVBARZE5N6ZZGSQPBCQQB5UWBQI3XHA, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.651959444 (3h28m9.691532564s) [noise=true , address=77.90.31.15:4698 , piece_id=QUFN2FMBZ7LEHUG4KGFJAK2SH3AXC5IWZXPJWVU5J4ZIAMKXBG4A, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.652208931 (3h28m9.691782051s) [noise=true , address=45.91.116.7:4009 , piece_id=QUXR35LPTYERLQFFRSPDFQJSL4ZLG3UC24TLAO3KM7GWIIXVMH5A, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.652447387 (3h28m9.692020508s) [noise=true , address=45.91.116.14:4190 , piece_id=YUS2SSQ3X4EJWB4C2HMULCEI4HQXOZYK72PDPKTQEZLRQZVIRIGA, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.652731360 (3h28m9.692304481s) [noise=true , address=77.90.31.16:4734 , piece_id=TDZ5HN6FOC2DVU3MQEXWEK6DMI2CEWDBFQFXUWLSCK4JD2JXXBJA, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.653055210 (3h28m9.69262833s) [noise=true , address=45.91.116.6:4974 , piece_id=5IU6JHBQ55BM2CF55RM3CQAAHIUJC7RKFTQOCEWAOCBRHUXGE3RA, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.653382968 (3h28m9.692956078s) [noise=true , address=77.90.31.18:4807 , piece_id=PBRG7U66P3LKM3DD2TG5RANULZTEWWHUA6WGG6T6BGPLFNH5RD2Q, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.653599151 (3h28m9.693172271s) [noise=true , address=77.90.31.19:4841 , piece_id=2PKDYTCWZD3N6SB2XQORTLAYWZMI4QVBVB2BLJJAED3LYBNWT2AA, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.653863287 (3h28m9.693436407s) [noise=true , address=45.91.116.10:4082 , piece_id=P2TPQBFLBS3XA6ZRRURO3LM6IQNRHANQNOCSMHUG6EYGWVZLL6YQ, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.654064993 (3h28m9.693638113s) [noise=true , address=45.91.116.4:4938 , piece_id=M5ZS5QZRPBMBOSLT2KTFY7PF72NBOCG2LSTDMBV3QQL3UMLOL6MQ, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.654251269 (3h28m9.693824389s) [noise=true , address=45.91.116.13:4155 , piece_id=P4OLS7LLAU5OEV7P4NNU7L53OFRCZUROLA4HGMXPPMQWTYZB2O3Q, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.654432817 (3h28m9.694005937s) [noise=true , address=45.91.116.12:4226 , piece_id=FTFHJJ27WCDYQBKMRYQXXGFKKCXUIXO5TYZKXCGUYQIUDYDJUVGA, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:33.654639201 (3h28m9.694212311s) [noise=true , address=45.91.116.8:4047 , piece_id=YNL66X7DUB32UU2ZNPWLHIF4Y3HRPK3JUF4ECXAFZFQ3BTXH52KA, seg_pos={0 0}, upload=3020]: Uploading piece... Sep 6 16:56:35.469006996 (3h28m11.508580116s) [upload=3020]: Next returned nil segment Sep 6 16:56:35.469024489 (3h28m11.508597609s) [upload=3020]: Waiting for error group managing segments... Sep 6 16:56:35.773140196 (3h28m11.812713316s) [noise=true , address=77.90.31.15:4698 , piece_id=QUFN2FMBZ7LEHUG4KGFJAK2SH3AXC5IWZXPJWVU5J4ZIAMKXBG4A, seg_pos={0 0}, upload=3020]: Done uploading piece. err: <nil> Sep 6 16:56:35.790480969 (3h28m11.830054089s) [noise=true , address=45.91.116.13:4155 , piece_id=P4OLS7LLAU5OEV7P4NNU7L53OFRCZUROLA4HGMXPPMQWTYZB2O3Q, seg_pos={0 0}, upload=3020]: Done uploading piece. err: <nil> Sep 6 16:56:35.802972712 (3h28m11.842545832s) [noise=true , address=45.91.116.5:4297 , piece_id=WM7LU5DMM5TIAOYGE6VWCLIQ7QCGLDUTA6FH3DWEEW34GI62CTDQ, seg_pos={0 0}, upload=3020]: Done uploading piece. err: <nil> Sep 6 16:56:35.864354275 (3h28m11.903927395s) [noise=true , address=45.91.116.7:4009 , piece_id=QUXR35LPTYERLQFFRSPDFQJSL4ZLG3UC24TLAO3KM7GWIIXVMH5A, seg_pos={0 0}, upload=3020]: Done uploading piece. err: <nil> Sep 6 16:56:35.873268152 (3h28m11.912841262s) [noise=true , address=45.91.116.5:4299 , piece_id=UGYUAYFSFZN472CFWWGB5ITJLPHOZS7X6B3DV7MPN2CHE5KN4GAQ, seg_pos={0 0}, upload=3020]: Done uploading piece. err: <nil> Sep 6 16:56:35.917053294 (3h28m11.956626414s) [noise=true , address=45.91.116.8:4047 , piece_id=YNL66X7DUB32UU2ZNPWLHIF4Y3HRPK3JUF4ECXAFZFQ3BTXH52KA, seg_pos={0 0}, upload=3020]: Done uploading piece. err: <nil> Sep 6 16:56:35.927307436 (3h28m11.966880556s) [noise=true , address=45.91.116.10:4082 , piece_id=P2TPQBFLBS3XA6ZRRURO3LM6IQNRHANQNOCSMHUG6EYGWVZLL6YQ, seg_pos={0 0}, upload=3020]: Done uploading piece. err: <nil> Sep 6 16:56:35.931391341 (3h28m11.970964461s) [noise=true , address=77.90.31.16:4734 , piece_id=TDZ5HN6FOC2DVU3MQEXWEK6DMI2CEWDBFQFXUWLSCK4JD2JXXBJA, seg_pos={0 0}, upload=3020]: Done uploading piece. err: <nil> Sep 6 16:56:35.931467917 (3h28m11.971041037s) [noise=true , address=45.91.116.12:4225 , piece_id=IVQZNOHCINPMDXW6MSQA5HBWPYSNIGWCTLRNRV374M5OCNBBNM4A, seg_pos={0 0}, upload=3020]: Done uploading piece. err: <nil> Sep 6 16:56:35.978933703 (3h28m12.018506824s) [noise=true , address=45.91.116.6:4974 , piece_id=5IU6JHBQ55BM2CF55RM3CQAAHIUJC7RKFTQOCEWAOCBRHUXGE3RA, seg_pos={0 0}, upload=3020]: Done uploading piece. err: <nil> Sep 6 16:56:36.017688511 (3h28m12.057261631s) [noise=true , address=45.91.116.4:4938 , piece_id=M5ZS5QZRPBMBOSLT2KTFY7PF72NBOCG2LSTDMBV3QQL3UMLOL6MQ, seg_pos={0 0}, upload=3020]: Done uploading piece. err: <nil> Sep 6 16:56:36.038924416 (3h28m12.078497536s) [noise=true , address=45.91.116.14:4190 , piece_id=YUS2SSQ3X4EJWB4C2HMULCEI4HQXOZYK72PDPKTQEZLRQZVIRIGA, seg_pos={0 0}, upload=3020]: Done uploading piece. err: <nil> Sep 6 16:56:36.042317719 (3h28m12.081890839s) [noise=true , address=45.91.116.8:4045 , piece_id=KMC7WJE6DAYEK5X6R2ZGGZVBARZE5N6ZZGSQPBCQQB5UWBQI3XHA, seg_pos={0 0}, upload=3020]: Done uploading piece. err: <nil> Sep 6 16:56:36.042329773 (3h28m12.081902883s) [seg_pos={0 0}, upload=3020]: Segment reached optimal threshold of 13 pieces. Sep 6 16:56:36.042478116 (3h28m12.082051226s) [noise=true , address=77.90.31.18:4807 , piece_id=PBRG7U66P3LKM3DD2TG5RANULZTEWWHUA6WGG6T6BGPLFNH5RD2Q, seg_pos={0 0}, upload=3020]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS): protocol: expected piece hash; context canceled; EOF Sep 6 16:56:36.042624676 (3h28m12.082197796s) [noise=true , address=77.90.31.19:4841 , piece_id=2PKDYTCWZD3N6SB2XQORTLAYWZMI4QVBVB2BLJJAED3LYBNWT2AA, seg_pos={0 0}, upload=3020]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:35860->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:35860->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35860->77.90.31.19:4841: use of closed network connection Sep 6 16:56:36.042639154 (3h28m12.082212274s) [noise=true , address=45.91.116.12:4226 , piece_id=FTFHJJ27WCDYQBKMRYQXXGFKKCXUIXO5TYZKXCGUYQIUDYDJUVGA, seg_pos={0 0}, upload=3020]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: write tcp 188.166.225.170:39776->45.91.116.12:4226: use of closed network connection; write tcp 188.166.225.170:39776->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39776->45.91.116.12:4226: use of closed network connection Sep 6 16:56:36.042659133 (3h28m12.082232253s) [seg_pos={0 0}, upload=3020]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 16:56:36.042722323 (3h28m12.082295444s) [upload=3020]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 16:56:36.042726211 (3h28m12.082299331s) [upload=3020]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 16:56:36.355029354 (3h28m12.394602474s) [upload=3020]: Done uploading segments... Sep 6 16:56:36.355046867 (3h28m12.394619977s) [upload=3020]: Upload finished. err: <nil> Duration: 30.001250053s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 24 out of 60 is failed (threshold=3) [XXXX.XX..X..X...XX..XX..X.X.XX....XX.XXX.....X..X....X......] duration: 30.00198292s FAIL (2025-09-06 16:56:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 16:51:40.468819831 (3h23m16.508392951s) [upload=2948]: Starting upload Sep 6 16:51:40.468909502 (3h23m16.508482622s) [upload=2948]: Done starting upload Sep 6 16:51:40.469143449 (3h23m16.508716569s) [upload=2948]: Uploading segments... Sep 6 16:51:40.469186472 (3h23m16.508759592s) [upload=2948]: Got next segment. Inline: false Sep 6 16:51:40.469214385 (3h23m16.508787505s) [upload=2948]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 16:51:40.469219545 (3h23m16.508792665s) [upload=2948]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 16:51:40.720763625 (3h23m16.760336745s) [seg_pos={0 0}, upload=2948]: Begin upload segment... Sep 6 16:51:40.720803491 (3h23m16.760376611s) [seg_pos={0 0}, upload=2948]: Done begin upload segment. Sep 6 16:51:40.720823830 (3h23m16.76039694s) [noise=true , address=45.91.116.9:4263 , piece_id=ACUXJWD4GS7RGXUYPF4RMPTUMVOLYAZ645TINXVGDTGBOWBLBWHA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.720847756 (3h23m16.760420876s) [noise=true , address=77.90.31.19:4843 , piece_id=KBVEXS3RGJE6NG2NDCLI4DJMNIVCIDZGN3XBMJ32QLNW4Q42TMBQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721197264 (3h23m16.760770374s) [noise=true , address=45.91.116.4:4937 , piece_id=ELDRT4NUA5EDMPNLRQDRISVEIM7BGBCCKZBCKSLHUEWMUVZL7AMQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721222492 (3h23m16.760795602s) [noise=true , address=45.91.116.4:4939 , piece_id=6EIHMKXQQEBNFFYHZNJ23GARMCEQQWOJPHWBD4W4USVJKR7JX4NQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721479014 (3h23m16.761052134s) [noise=true , address=45.91.116.6:4974 , piece_id=3QR5JAL7G4OMPA34QZ57MWNKNOWBPSWNDCKROLWOGXMWALO2RPRQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721570098 (3h23m16.761143218s) [noise=true , address=45.91.116.13:4155 , piece_id=4TOH2H4GGC24RIWZF6NMLLLRTBHBGT2BH3IEGZP6CWFKPWCHPL7Q, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721856937 (3h23m16.761430057s) [noise=true , address=77.90.31.14:4662 , piece_id=2HPYC52YRN3FXIAZMKCHMA4AYKZQ5XUIDMXNMAMMOTFUWTCK3VQQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722015730 (3h23m16.76158885s) [noise=true , address=45.91.116.12:4225 , piece_id=KI4LWOQPSV2YZKRXGLWL6GORFJXNPII2RE4TW4U6BJQCU4HRH4NQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722351934 (3h23m16.761925044s) [noise=true , address=45.91.116.5:4297 , piece_id=DU7L2DRLWMLV5OF4FJSVJJ4GIUMWPLQSVDMJVLAIFBE5N3BST6HA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722409674 (3h23m16.761982875s) [noise=true , address=45.91.116.9:4262 , piece_id=LRNRJRYK4LHOILTVMG3RWZ35LTAY24EJALOBF2HWV7AVERS2AGXQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722585891 (3h23m16.762159011s) [noise=true , address=77.90.31.12:4590 , piece_id=5YFKDFUC2CE4KBXYIRC2M55I6TDLJVX3OE4KRLCNNTN3YWOUKFXA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722697465 (3h23m16.762270585s) [noise=true , address=45.91.116.11:4117 , piece_id=5C6V6XTCDPGU2NWZDGSWKREXJBLAJY75XD2XPMDDZU7I5VUPKULQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722810080 (3h23m16.76238319s) [noise=true , address=45.91.116.6:4973 , piece_id=ERHKLODTTAWM6KJJAU76RF2REZMPNEE3UULW2E77DPPT3YEYQFXQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722943706 (3h23m16.762516826s) [noise=true , address=45.91.116.11:4118 , piece_id=XMPTB4CJVYKVSKIQZAPDNJERAGYVDS23ALCULMSZ6MQC65YJML2Q, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.723025482 (3h23m16.762598592s) [noise=true , address=77.90.31.10:4517 , piece_id=UKYCK3PNW73SR2ETCDA6BGJX7EBNRUNPAYTPZSS54BOGUK7KNERA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.723265752 (3h23m16.762838872s) [noise=true , address=45.91.116.10:4083 , piece_id=QNNCGHVWM6ZKDE7VHKH45AXSMSTWR6F4KKNBSRJSRYJQRYLYDCAQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:42.526679200 (3h23m18.56625237s) [upload=2948]: Next returned nil segment Sep 6 16:51:42.526699318 (3h23m18.566272449s) [upload=2948]: Waiting for error group managing segments... Sep 6 16:51:42.823705359 (3h23m18.863278479s) [noise=true , address=45.91.116.6:4973 , piece_id=ERHKLODTTAWM6KJJAU76RF2REZMPNEE3UULW2E77DPPT3YEYQFXQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.840145057 (3h23m18.879718178s) [noise=true , address=45.91.116.10:4083 , piece_id=QNNCGHVWM6ZKDE7VHKH45AXSMSTWR6F4KKNBSRJSRYJQRYLYDCAQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.866065405 (3h23m18.905638525s) [noise=true , address=45.91.116.12:4225 , piece_id=KI4LWOQPSV2YZKRXGLWL6GORFJXNPII2RE4TW4U6BJQCU4HRH4NQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.872872561 (3h23m18.912445681s) [noise=true , address=45.91.116.9:4263 , piece_id=ACUXJWD4GS7RGXUYPF4RMPTUMVOLYAZ645TINXVGDTGBOWBLBWHA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.915160650 (3h23m18.95473377s) [noise=true , address=45.91.116.5:4297 , piece_id=DU7L2DRLWMLV5OF4FJSVJJ4GIUMWPLQSVDMJVLAIFBE5N3BST6HA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.932976680 (3h23m18.9725498s) [noise=true , address=77.90.31.14:4662 , piece_id=2HPYC52YRN3FXIAZMKCHMA4AYKZQ5XUIDMXNMAMMOTFUWTCK3VQQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.960547097 (3h23m19.000120227s) [noise=true , address=45.91.116.11:4117 , piece_id=5C6V6XTCDPGU2NWZDGSWKREXJBLAJY75XD2XPMDDZU7I5VUPKULQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.999827930 (3h23m19.03940105s) [noise=true , address=45.91.116.4:4937 , piece_id=ELDRT4NUA5EDMPNLRQDRISVEIM7BGBCCKZBCKSLHUEWMUVZL7AMQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.032435255 (3h23m19.072008375s) [noise=true , address=45.91.116.9:4262 , piece_id=LRNRJRYK4LHOILTVMG3RWZ35LTAY24EJALOBF2HWV7AVERS2AGXQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.045906972 (3h23m19.085480092s) [noise=true , address=77.90.31.19:4843 , piece_id=KBVEXS3RGJE6NG2NDCLI4DJMNIVCIDZGN3XBMJ32QLNW4Q42TMBQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.050794755 (3h23m19.090367875s) [noise=true , address=45.91.116.13:4155 , piece_id=4TOH2H4GGC24RIWZF6NMLLLRTBHBGT2BH3IEGZP6CWFKPWCHPL7Q, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.056786971 (3h23m19.096360091s) [noise=true , address=77.90.31.12:4590 , piece_id=5YFKDFUC2CE4KBXYIRC2M55I6TDLJVX3OE4KRLCNNTN3YWOUKFXA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.063295826 (3h23m19.102868946s) [noise=true , address=45.91.116.11:4118 , piece_id=XMPTB4CJVYKVSKIQZAPDNJERAGYVDS23ALCULMSZ6MQC65YJML2Q, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.063314001 (3h23m19.102887121s) [seg_pos={0 0}, upload=2948]: Segment reached optimal threshold of 13 pieces. Sep 6 16:51:43.063542228 (3h23m19.103115348s) [noise=true , address=77.90.31.10:4517 , piece_id=UKYCK3PNW73SR2ETCDA6BGJX7EBNRUNPAYTPZSS54BOGUK7KNERA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: write tcp 188.166.225.170:59286->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:59286->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59286->77.90.31.10:4517: use of closed network connection Sep 6 16:51:43.063625216 (3h23m19.103198336s) [noise=true , address=45.91.116.6:4974 , piece_id=3QR5JAL7G4OMPA34QZ57MWNKNOWBPSWNDCKROLWOGXMWALO2RPRQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 6 16:51:43.063699328 (3h23m19.103272448s) [noise=true , address=45.91.116.4:4939 , piece_id=6EIHMKXQQEBNFFYHZNJ23GARMCEQQWOJPHWBD4W4USVJKR7JX4NQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 188.166.225.170:52570->45.91.116.4:4939: use of closed network connection; context canceled Sep 6 16:51:43.063710269 (3h23m19.103283389s) [seg_pos={0 0}, upload=2948]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 16:51:43.063780925 (3h23m19.103354045s) [upload=2948]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 16:51:43.063785022 (3h23m19.103358142s) [upload=2948]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 16:51:43.366135850 (3h23m19.40570897s) [upload=2948]: Done uploading segments... Sep 6 16:51:43.366155037 (3h23m19.405728157s) [upload=2948]: Upload finished. err: <nil> Duration: 30.002006877s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.....X.........] duration: 644.839415ms remote: "tcp:13.233.175.166:443" response headers: X-Cluster-Name=ap-south-1-prod-hosting-red Date=Sat, 06 Sep 2025 16:49:03 GMT Content-Type=text/html Content-Length=164 target: https://www.storj.io/dcs-satellites FAIL (2025-09-06 16:56:12 UTC) orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n" Duration: 644.867408ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [XXXXX.XX..X..X...XX..XX..X.X.XX....XX.XXX.....X..X....X.....] duration: 6.462093797s FAIL (2025-09-06 16:55:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 16:51:40.468819831 (3h23m16.508392951s) [upload=2948]: Starting upload Sep 6 16:51:40.468909502 (3h23m16.508482622s) [upload=2948]: Done starting upload Sep 6 16:51:40.469143449 (3h23m16.508716569s) [upload=2948]: Uploading segments... Sep 6 16:51:40.469186472 (3h23m16.508759592s) [upload=2948]: Got next segment. Inline: false Sep 6 16:51:40.469214385 (3h23m16.508787505s) [upload=2948]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 16:51:40.469219545 (3h23m16.508792665s) [upload=2948]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 16:51:40.720763625 (3h23m16.760336745s) [seg_pos={0 0}, upload=2948]: Begin upload segment... Sep 6 16:51:40.720803491 (3h23m16.760376611s) [seg_pos={0 0}, upload=2948]: Done begin upload segment. Sep 6 16:51:40.720823830 (3h23m16.76039694s) [noise=true , address=45.91.116.9:4263 , piece_id=ACUXJWD4GS7RGXUYPF4RMPTUMVOLYAZ645TINXVGDTGBOWBLBWHA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.720847756 (3h23m16.760420876s) [noise=true , address=77.90.31.19:4843 , piece_id=KBVEXS3RGJE6NG2NDCLI4DJMNIVCIDZGN3XBMJ32QLNW4Q42TMBQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721197264 (3h23m16.760770374s) [noise=true , address=45.91.116.4:4937 , piece_id=ELDRT4NUA5EDMPNLRQDRISVEIM7BGBCCKZBCKSLHUEWMUVZL7AMQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721222492 (3h23m16.760795602s) [noise=true , address=45.91.116.4:4939 , piece_id=6EIHMKXQQEBNFFYHZNJ23GARMCEQQWOJPHWBD4W4USVJKR7JX4NQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721479014 (3h23m16.761052134s) [noise=true , address=45.91.116.6:4974 , piece_id=3QR5JAL7G4OMPA34QZ57MWNKNOWBPSWNDCKROLWOGXMWALO2RPRQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721570098 (3h23m16.761143218s) [noise=true , address=45.91.116.13:4155 , piece_id=4TOH2H4GGC24RIWZF6NMLLLRTBHBGT2BH3IEGZP6CWFKPWCHPL7Q, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721856937 (3h23m16.761430057s) [noise=true , address=77.90.31.14:4662 , piece_id=2HPYC52YRN3FXIAZMKCHMA4AYKZQ5XUIDMXNMAMMOTFUWTCK3VQQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722015730 (3h23m16.76158885s) [noise=true , address=45.91.116.12:4225 , piece_id=KI4LWOQPSV2YZKRXGLWL6GORFJXNPII2RE4TW4U6BJQCU4HRH4NQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722351934 (3h23m16.761925044s) [noise=true , address=45.91.116.5:4297 , piece_id=DU7L2DRLWMLV5OF4FJSVJJ4GIUMWPLQSVDMJVLAIFBE5N3BST6HA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722409674 (3h23m16.761982875s) [noise=true , address=45.91.116.9:4262 , piece_id=LRNRJRYK4LHOILTVMG3RWZ35LTAY24EJALOBF2HWV7AVERS2AGXQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722585891 (3h23m16.762159011s) [noise=true , address=77.90.31.12:4590 , piece_id=5YFKDFUC2CE4KBXYIRC2M55I6TDLJVX3OE4KRLCNNTN3YWOUKFXA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722697465 (3h23m16.762270585s) [noise=true , address=45.91.116.11:4117 , piece_id=5C6V6XTCDPGU2NWZDGSWKREXJBLAJY75XD2XPMDDZU7I5VUPKULQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722810080 (3h23m16.76238319s) [noise=true , address=45.91.116.6:4973 , piece_id=ERHKLODTTAWM6KJJAU76RF2REZMPNEE3UULW2E77DPPT3YEYQFXQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722943706 (3h23m16.762516826s) [noise=true , address=45.91.116.11:4118 , piece_id=XMPTB4CJVYKVSKIQZAPDNJERAGYVDS23ALCULMSZ6MQC65YJML2Q, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.723025482 (3h23m16.762598592s) [noise=true , address=77.90.31.10:4517 , piece_id=UKYCK3PNW73SR2ETCDA6BGJX7EBNRUNPAYTPZSS54BOGUK7KNERA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.723265752 (3h23m16.762838872s) [noise=true , address=45.91.116.10:4083 , piece_id=QNNCGHVWM6ZKDE7VHKH45AXSMSTWR6F4KKNBSRJSRYJQRYLYDCAQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:42.526679200 (3h23m18.56625237s) [upload=2948]: Next returned nil segment Sep 6 16:51:42.526699318 (3h23m18.566272449s) [upload=2948]: Waiting for error group managing segments... Sep 6 16:51:42.823705359 (3h23m18.863278479s) [noise=true , address=45.91.116.6:4973 , piece_id=ERHKLODTTAWM6KJJAU76RF2REZMPNEE3UULW2E77DPPT3YEYQFXQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.840145057 (3h23m18.879718178s) [noise=true , address=45.91.116.10:4083 , piece_id=QNNCGHVWM6ZKDE7VHKH45AXSMSTWR6F4KKNBSRJSRYJQRYLYDCAQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.866065405 (3h23m18.905638525s) [noise=true , address=45.91.116.12:4225 , piece_id=KI4LWOQPSV2YZKRXGLWL6GORFJXNPII2RE4TW4U6BJQCU4HRH4NQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.872872561 (3h23m18.912445681s) [noise=true , address=45.91.116.9:4263 , piece_id=ACUXJWD4GS7RGXUYPF4RMPTUMVOLYAZ645TINXVGDTGBOWBLBWHA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.915160650 (3h23m18.95473377s) [noise=true , address=45.91.116.5:4297 , piece_id=DU7L2DRLWMLV5OF4FJSVJJ4GIUMWPLQSVDMJVLAIFBE5N3BST6HA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.932976680 (3h23m18.9725498s) [noise=true , address=77.90.31.14:4662 , piece_id=2HPYC52YRN3FXIAZMKCHMA4AYKZQ5XUIDMXNMAMMOTFUWTCK3VQQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.960547097 (3h23m19.000120227s) [noise=true , address=45.91.116.11:4117 , piece_id=5C6V6XTCDPGU2NWZDGSWKREXJBLAJY75XD2XPMDDZU7I5VUPKULQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.999827930 (3h23m19.03940105s) [noise=true , address=45.91.116.4:4937 , piece_id=ELDRT4NUA5EDMPNLRQDRISVEIM7BGBCCKZBCKSLHUEWMUVZL7AMQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.032435255 (3h23m19.072008375s) [noise=true , address=45.91.116.9:4262 , piece_id=LRNRJRYK4LHOILTVMG3RWZ35LTAY24EJALOBF2HWV7AVERS2AGXQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.045906972 (3h23m19.085480092s) [noise=true , address=77.90.31.19:4843 , piece_id=KBVEXS3RGJE6NG2NDCLI4DJMNIVCIDZGN3XBMJ32QLNW4Q42TMBQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.050794755 (3h23m19.090367875s) [noise=true , address=45.91.116.13:4155 , piece_id=4TOH2H4GGC24RIWZF6NMLLLRTBHBGT2BH3IEGZP6CWFKPWCHPL7Q, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.056786971 (3h23m19.096360091s) [noise=true , address=77.90.31.12:4590 , piece_id=5YFKDFUC2CE4KBXYIRC2M55I6TDLJVX3OE4KRLCNNTN3YWOUKFXA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.063295826 (3h23m19.102868946s) [noise=true , address=45.91.116.11:4118 , piece_id=XMPTB4CJVYKVSKIQZAPDNJERAGYVDS23ALCULMSZ6MQC65YJML2Q, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.063314001 (3h23m19.102887121s) [seg_pos={0 0}, upload=2948]: Segment reached optimal threshold of 13 pieces. Sep 6 16:51:43.063542228 (3h23m19.103115348s) [noise=true , address=77.90.31.10:4517 , piece_id=UKYCK3PNW73SR2ETCDA6BGJX7EBNRUNPAYTPZSS54BOGUK7KNERA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: write tcp 188.166.225.170:59286->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:59286->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59286->77.90.31.10:4517: use of closed network connection Sep 6 16:51:43.063625216 (3h23m19.103198336s) [noise=true , address=45.91.116.6:4974 , piece_id=3QR5JAL7G4OMPA34QZ57MWNKNOWBPSWNDCKROLWOGXMWALO2RPRQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 6 16:51:43.063699328 (3h23m19.103272448s) [noise=true , address=45.91.116.4:4939 , piece_id=6EIHMKXQQEBNFFYHZNJ23GARMCEQQWOJPHWBD4W4USVJKR7JX4NQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 188.166.225.170:52570->45.91.116.4:4939: use of closed network connection; context canceled Sep 6 16:51:43.063710269 (3h23m19.103283389s) [seg_pos={0 0}, upload=2948]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 16:51:43.063780925 (3h23m19.103354045s) [upload=2948]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 16:51:43.063785022 (3h23m19.103358142s) [upload=2948]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 16:51:43.366135850 (3h23m19.40570897s) [upload=2948]: Done uploading segments... Sep 6 16:51:43.366155037 (3h23m19.405728157s) [upload=2948]: Upload finished. err: <nil> Duration: 6.462122604s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [......X........] duration: 634.91294ms remote: "tcp:13.233.175.166:443" response headers: X-Cluster-Name=ap-south-1-prod-hosting-red Date=Sat, 06 Sep 2025 16:49:03 GMT Content-Type=text/html Content-Length=164 target: https://www.storj.io/dcs-satellites FAIL (2025-09-06 16:55:15 UTC) orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n" Duration: 634.93893ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [.XXXXX.XX..X..X...XX..XX..X.X.XX....XX.XXX.....X..X....X....] duration: 16.632212831s FAIL (2025-09-06 16:55:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 16:51:40.468819831 (3h23m16.508392951s) [upload=2948]: Starting upload Sep 6 16:51:40.468909502 (3h23m16.508482622s) [upload=2948]: Done starting upload Sep 6 16:51:40.469143449 (3h23m16.508716569s) [upload=2948]: Uploading segments... Sep 6 16:51:40.469186472 (3h23m16.508759592s) [upload=2948]: Got next segment. Inline: false Sep 6 16:51:40.469214385 (3h23m16.508787505s) [upload=2948]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 16:51:40.469219545 (3h23m16.508792665s) [upload=2948]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 16:51:40.720763625 (3h23m16.760336745s) [seg_pos={0 0}, upload=2948]: Begin upload segment... Sep 6 16:51:40.720803491 (3h23m16.760376611s) [seg_pos={0 0}, upload=2948]: Done begin upload segment. Sep 6 16:51:40.720823830 (3h23m16.76039694s) [noise=true , address=45.91.116.9:4263 , piece_id=ACUXJWD4GS7RGXUYPF4RMPTUMVOLYAZ645TINXVGDTGBOWBLBWHA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.720847756 (3h23m16.760420876s) [noise=true , address=77.90.31.19:4843 , piece_id=KBVEXS3RGJE6NG2NDCLI4DJMNIVCIDZGN3XBMJ32QLNW4Q42TMBQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721197264 (3h23m16.760770374s) [noise=true , address=45.91.116.4:4937 , piece_id=ELDRT4NUA5EDMPNLRQDRISVEIM7BGBCCKZBCKSLHUEWMUVZL7AMQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721222492 (3h23m16.760795602s) [noise=true , address=45.91.116.4:4939 , piece_id=6EIHMKXQQEBNFFYHZNJ23GARMCEQQWOJPHWBD4W4USVJKR7JX4NQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721479014 (3h23m16.761052134s) [noise=true , address=45.91.116.6:4974 , piece_id=3QR5JAL7G4OMPA34QZ57MWNKNOWBPSWNDCKROLWOGXMWALO2RPRQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721570098 (3h23m16.761143218s) [noise=true , address=45.91.116.13:4155 , piece_id=4TOH2H4GGC24RIWZF6NMLLLRTBHBGT2BH3IEGZP6CWFKPWCHPL7Q, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721856937 (3h23m16.761430057s) [noise=true , address=77.90.31.14:4662 , piece_id=2HPYC52YRN3FXIAZMKCHMA4AYKZQ5XUIDMXNMAMMOTFUWTCK3VQQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722015730 (3h23m16.76158885s) [noise=true , address=45.91.116.12:4225 , piece_id=KI4LWOQPSV2YZKRXGLWL6GORFJXNPII2RE4TW4U6BJQCU4HRH4NQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722351934 (3h23m16.761925044s) [noise=true , address=45.91.116.5:4297 , piece_id=DU7L2DRLWMLV5OF4FJSVJJ4GIUMWPLQSVDMJVLAIFBE5N3BST6HA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722409674 (3h23m16.761982875s) [noise=true , address=45.91.116.9:4262 , piece_id=LRNRJRYK4LHOILTVMG3RWZ35LTAY24EJALOBF2HWV7AVERS2AGXQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722585891 (3h23m16.762159011s) [noise=true , address=77.90.31.12:4590 , piece_id=5YFKDFUC2CE4KBXYIRC2M55I6TDLJVX3OE4KRLCNNTN3YWOUKFXA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722697465 (3h23m16.762270585s) [noise=true , address=45.91.116.11:4117 , piece_id=5C6V6XTCDPGU2NWZDGSWKREXJBLAJY75XD2XPMDDZU7I5VUPKULQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722810080 (3h23m16.76238319s) [noise=true , address=45.91.116.6:4973 , piece_id=ERHKLODTTAWM6KJJAU76RF2REZMPNEE3UULW2E77DPPT3YEYQFXQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722943706 (3h23m16.762516826s) [noise=true , address=45.91.116.11:4118 , piece_id=XMPTB4CJVYKVSKIQZAPDNJERAGYVDS23ALCULMSZ6MQC65YJML2Q, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.723025482 (3h23m16.762598592s) [noise=true , address=77.90.31.10:4517 , piece_id=UKYCK3PNW73SR2ETCDA6BGJX7EBNRUNPAYTPZSS54BOGUK7KNERA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.723265752 (3h23m16.762838872s) [noise=true , address=45.91.116.10:4083 , piece_id=QNNCGHVWM6ZKDE7VHKH45AXSMSTWR6F4KKNBSRJSRYJQRYLYDCAQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:42.526679200 (3h23m18.56625237s) [upload=2948]: Next returned nil segment Sep 6 16:51:42.526699318 (3h23m18.566272449s) [upload=2948]: Waiting for error group managing segments... Sep 6 16:51:42.823705359 (3h23m18.863278479s) [noise=true , address=45.91.116.6:4973 , piece_id=ERHKLODTTAWM6KJJAU76RF2REZMPNEE3UULW2E77DPPT3YEYQFXQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.840145057 (3h23m18.879718178s) [noise=true , address=45.91.116.10:4083 , piece_id=QNNCGHVWM6ZKDE7VHKH45AXSMSTWR6F4KKNBSRJSRYJQRYLYDCAQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.866065405 (3h23m18.905638525s) [noise=true , address=45.91.116.12:4225 , piece_id=KI4LWOQPSV2YZKRXGLWL6GORFJXNPII2RE4TW4U6BJQCU4HRH4NQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.872872561 (3h23m18.912445681s) [noise=true , address=45.91.116.9:4263 , piece_id=ACUXJWD4GS7RGXUYPF4RMPTUMVOLYAZ645TINXVGDTGBOWBLBWHA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.915160650 (3h23m18.95473377s) [noise=true , address=45.91.116.5:4297 , piece_id=DU7L2DRLWMLV5OF4FJSVJJ4GIUMWPLQSVDMJVLAIFBE5N3BST6HA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.932976680 (3h23m18.9725498s) [noise=true , address=77.90.31.14:4662 , piece_id=2HPYC52YRN3FXIAZMKCHMA4AYKZQ5XUIDMXNMAMMOTFUWTCK3VQQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.960547097 (3h23m19.000120227s) [noise=true , address=45.91.116.11:4117 , piece_id=5C6V6XTCDPGU2NWZDGSWKREXJBLAJY75XD2XPMDDZU7I5VUPKULQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.999827930 (3h23m19.03940105s) [noise=true , address=45.91.116.4:4937 , piece_id=ELDRT4NUA5EDMPNLRQDRISVEIM7BGBCCKZBCKSLHUEWMUVZL7AMQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.032435255 (3h23m19.072008375s) [noise=true , address=45.91.116.9:4262 , piece_id=LRNRJRYK4LHOILTVMG3RWZ35LTAY24EJALOBF2HWV7AVERS2AGXQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.045906972 (3h23m19.085480092s) [noise=true , address=77.90.31.19:4843 , piece_id=KBVEXS3RGJE6NG2NDCLI4DJMNIVCIDZGN3XBMJ32QLNW4Q42TMBQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.050794755 (3h23m19.090367875s) [noise=true , address=45.91.116.13:4155 , piece_id=4TOH2H4GGC24RIWZF6NMLLLRTBHBGT2BH3IEGZP6CWFKPWCHPL7Q, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.056786971 (3h23m19.096360091s) [noise=true , address=77.90.31.12:4590 , piece_id=5YFKDFUC2CE4KBXYIRC2M55I6TDLJVX3OE4KRLCNNTN3YWOUKFXA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.063295826 (3h23m19.102868946s) [noise=true , address=45.91.116.11:4118 , piece_id=XMPTB4CJVYKVSKIQZAPDNJERAGYVDS23ALCULMSZ6MQC65YJML2Q, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.063314001 (3h23m19.102887121s) [seg_pos={0 0}, upload=2948]: Segment reached optimal threshold of 13 pieces. Sep 6 16:51:43.063542228 (3h23m19.103115348s) [noise=true , address=77.90.31.10:4517 , piece_id=UKYCK3PNW73SR2ETCDA6BGJX7EBNRUNPAYTPZSS54BOGUK7KNERA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: write tcp 188.166.225.170:59286->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:59286->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59286->77.90.31.10:4517: use of closed network connection Sep 6 16:51:43.063625216 (3h23m19.103198336s) [noise=true , address=45.91.116.6:4974 , piece_id=3QR5JAL7G4OMPA34QZ57MWNKNOWBPSWNDCKROLWOGXMWALO2RPRQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 6 16:51:43.063699328 (3h23m19.103272448s) [noise=true , address=45.91.116.4:4939 , piece_id=6EIHMKXQQEBNFFYHZNJ23GARMCEQQWOJPHWBD4W4USVJKR7JX4NQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 188.166.225.170:52570->45.91.116.4:4939: use of closed network connection; context canceled Sep 6 16:51:43.063710269 (3h23m19.103283389s) [seg_pos={0 0}, upload=2948]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 16:51:43.063780925 (3h23m19.103354045s) [upload=2948]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 16:51:43.063785022 (3h23m19.103358142s) [upload=2948]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 16:51:43.366135850 (3h23m19.40570897s) [upload=2948]: Done uploading segments... Sep 6 16:51:43.366155037 (3h23m19.405728157s) [upload=2948]: Upload finished. err: <nil> Duration: 16.632240584s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.......X.......] duration: 732.997281ms remote: "tcp:13.233.175.166:443" response headers: X-Cluster-Name=ap-south-1-prod-hosting-red Date=Sat, 06 Sep 2025 16:49:03 GMT Content-Type=text/html Content-Length=164 target: https://www.storj.io/dcs-satellites FAIL (2025-09-06 16:54:24 UTC) orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n" Duration: 733.026046ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 26 out of 60 is failed (threshold=3) [X.XXXXX.XX..X..X...XX..XX..X.X.XX....XX.XXX.....X..X....X...] duration: 6.98545723s FAIL (2025-09-06 16:53:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 16:51:40.468819831 (3h23m16.508392951s) [upload=2948]: Starting upload Sep 6 16:51:40.468909502 (3h23m16.508482622s) [upload=2948]: Done starting upload Sep 6 16:51:40.469143449 (3h23m16.508716569s) [upload=2948]: Uploading segments... Sep 6 16:51:40.469186472 (3h23m16.508759592s) [upload=2948]: Got next segment. Inline: false Sep 6 16:51:40.469214385 (3h23m16.508787505s) [upload=2948]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 16:51:40.469219545 (3h23m16.508792665s) [upload=2948]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 16:51:40.720763625 (3h23m16.760336745s) [seg_pos={0 0}, upload=2948]: Begin upload segment... Sep 6 16:51:40.720803491 (3h23m16.760376611s) [seg_pos={0 0}, upload=2948]: Done begin upload segment. Sep 6 16:51:40.720823830 (3h23m16.76039694s) [noise=true , address=45.91.116.9:4263 , piece_id=ACUXJWD4GS7RGXUYPF4RMPTUMVOLYAZ645TINXVGDTGBOWBLBWHA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.720847756 (3h23m16.760420876s) [noise=true , address=77.90.31.19:4843 , piece_id=KBVEXS3RGJE6NG2NDCLI4DJMNIVCIDZGN3XBMJ32QLNW4Q42TMBQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721197264 (3h23m16.760770374s) [noise=true , address=45.91.116.4:4937 , piece_id=ELDRT4NUA5EDMPNLRQDRISVEIM7BGBCCKZBCKSLHUEWMUVZL7AMQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721222492 (3h23m16.760795602s) [noise=true , address=45.91.116.4:4939 , piece_id=6EIHMKXQQEBNFFYHZNJ23GARMCEQQWOJPHWBD4W4USVJKR7JX4NQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721479014 (3h23m16.761052134s) [noise=true , address=45.91.116.6:4974 , piece_id=3QR5JAL7G4OMPA34QZ57MWNKNOWBPSWNDCKROLWOGXMWALO2RPRQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721570098 (3h23m16.761143218s) [noise=true , address=45.91.116.13:4155 , piece_id=4TOH2H4GGC24RIWZF6NMLLLRTBHBGT2BH3IEGZP6CWFKPWCHPL7Q, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721856937 (3h23m16.761430057s) [noise=true , address=77.90.31.14:4662 , piece_id=2HPYC52YRN3FXIAZMKCHMA4AYKZQ5XUIDMXNMAMMOTFUWTCK3VQQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722015730 (3h23m16.76158885s) [noise=true , address=45.91.116.12:4225 , piece_id=KI4LWOQPSV2YZKRXGLWL6GORFJXNPII2RE4TW4U6BJQCU4HRH4NQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722351934 (3h23m16.761925044s) [noise=true , address=45.91.116.5:4297 , piece_id=DU7L2DRLWMLV5OF4FJSVJJ4GIUMWPLQSVDMJVLAIFBE5N3BST6HA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722409674 (3h23m16.761982875s) [noise=true , address=45.91.116.9:4262 , piece_id=LRNRJRYK4LHOILTVMG3RWZ35LTAY24EJALOBF2HWV7AVERS2AGXQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722585891 (3h23m16.762159011s) [noise=true , address=77.90.31.12:4590 , piece_id=5YFKDFUC2CE4KBXYIRC2M55I6TDLJVX3OE4KRLCNNTN3YWOUKFXA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722697465 (3h23m16.762270585s) [noise=true , address=45.91.116.11:4117 , piece_id=5C6V6XTCDPGU2NWZDGSWKREXJBLAJY75XD2XPMDDZU7I5VUPKULQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722810080 (3h23m16.76238319s) [noise=true , address=45.91.116.6:4973 , piece_id=ERHKLODTTAWM6KJJAU76RF2REZMPNEE3UULW2E77DPPT3YEYQFXQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722943706 (3h23m16.762516826s) [noise=true , address=45.91.116.11:4118 , piece_id=XMPTB4CJVYKVSKIQZAPDNJERAGYVDS23ALCULMSZ6MQC65YJML2Q, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.723025482 (3h23m16.762598592s) [noise=true , address=77.90.31.10:4517 , piece_id=UKYCK3PNW73SR2ETCDA6BGJX7EBNRUNPAYTPZSS54BOGUK7KNERA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.723265752 (3h23m16.762838872s) [noise=true , address=45.91.116.10:4083 , piece_id=QNNCGHVWM6ZKDE7VHKH45AXSMSTWR6F4KKNBSRJSRYJQRYLYDCAQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:42.526679200 (3h23m18.56625237s) [upload=2948]: Next returned nil segment Sep 6 16:51:42.526699318 (3h23m18.566272449s) [upload=2948]: Waiting for error group managing segments... Sep 6 16:51:42.823705359 (3h23m18.863278479s) [noise=true , address=45.91.116.6:4973 , piece_id=ERHKLODTTAWM6KJJAU76RF2REZMPNEE3UULW2E77DPPT3YEYQFXQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.840145057 (3h23m18.879718178s) [noise=true , address=45.91.116.10:4083 , piece_id=QNNCGHVWM6ZKDE7VHKH45AXSMSTWR6F4KKNBSRJSRYJQRYLYDCAQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.866065405 (3h23m18.905638525s) [noise=true , address=45.91.116.12:4225 , piece_id=KI4LWOQPSV2YZKRXGLWL6GORFJXNPII2RE4TW4U6BJQCU4HRH4NQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.872872561 (3h23m18.912445681s) [noise=true , address=45.91.116.9:4263 , piece_id=ACUXJWD4GS7RGXUYPF4RMPTUMVOLYAZ645TINXVGDTGBOWBLBWHA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.915160650 (3h23m18.95473377s) [noise=true , address=45.91.116.5:4297 , piece_id=DU7L2DRLWMLV5OF4FJSVJJ4GIUMWPLQSVDMJVLAIFBE5N3BST6HA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.932976680 (3h23m18.9725498s) [noise=true , address=77.90.31.14:4662 , piece_id=2HPYC52YRN3FXIAZMKCHMA4AYKZQ5XUIDMXNMAMMOTFUWTCK3VQQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.960547097 (3h23m19.000120227s) [noise=true , address=45.91.116.11:4117 , piece_id=5C6V6XTCDPGU2NWZDGSWKREXJBLAJY75XD2XPMDDZU7I5VUPKULQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.999827930 (3h23m19.03940105s) [noise=true , address=45.91.116.4:4937 , piece_id=ELDRT4NUA5EDMPNLRQDRISVEIM7BGBCCKZBCKSLHUEWMUVZL7AMQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.032435255 (3h23m19.072008375s) [noise=true , address=45.91.116.9:4262 , piece_id=LRNRJRYK4LHOILTVMG3RWZ35LTAY24EJALOBF2HWV7AVERS2AGXQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.045906972 (3h23m19.085480092s) [noise=true , address=77.90.31.19:4843 , piece_id=KBVEXS3RGJE6NG2NDCLI4DJMNIVCIDZGN3XBMJ32QLNW4Q42TMBQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.050794755 (3h23m19.090367875s) [noise=true , address=45.91.116.13:4155 , piece_id=4TOH2H4GGC24RIWZF6NMLLLRTBHBGT2BH3IEGZP6CWFKPWCHPL7Q, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.056786971 (3h23m19.096360091s) [noise=true , address=77.90.31.12:4590 , piece_id=5YFKDFUC2CE4KBXYIRC2M55I6TDLJVX3OE4KRLCNNTN3YWOUKFXA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.063295826 (3h23m19.102868946s) [noise=true , address=45.91.116.11:4118 , piece_id=XMPTB4CJVYKVSKIQZAPDNJERAGYVDS23ALCULMSZ6MQC65YJML2Q, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.063314001 (3h23m19.102887121s) [seg_pos={0 0}, upload=2948]: Segment reached optimal threshold of 13 pieces. Sep 6 16:51:43.063542228 (3h23m19.103115348s) [noise=true , address=77.90.31.10:4517 , piece_id=UKYCK3PNW73SR2ETCDA6BGJX7EBNRUNPAYTPZSS54BOGUK7KNERA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: write tcp 188.166.225.170:59286->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:59286->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59286->77.90.31.10:4517: use of closed network connection Sep 6 16:51:43.063625216 (3h23m19.103198336s) [noise=true , address=45.91.116.6:4974 , piece_id=3QR5JAL7G4OMPA34QZ57MWNKNOWBPSWNDCKROLWOGXMWALO2RPRQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 6 16:51:43.063699328 (3h23m19.103272448s) [noise=true , address=45.91.116.4:4939 , piece_id=6EIHMKXQQEBNFFYHZNJ23GARMCEQQWOJPHWBD4W4USVJKR7JX4NQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 188.166.225.170:52570->45.91.116.4:4939: use of closed network connection; context canceled Sep 6 16:51:43.063710269 (3h23m19.103283389s) [seg_pos={0 0}, upload=2948]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 16:51:43.063780925 (3h23m19.103354045s) [upload=2948]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 16:51:43.063785022 (3h23m19.103358142s) [upload=2948]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 16:51:43.366135850 (3h23m19.40570897s) [upload=2948]: Done uploading segments... Sep 6 16:51:43.366155037 (3h23m19.405728157s) [upload=2948]: Upload finished. err: <nil> Duration: 6.985488791s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [........X......] duration: 616.682341ms remote: "tcp:13.233.175.166:443" response headers: X-Cluster-Name=ap-south-1-prod-hosting-red Date=Sat, 06 Sep 2025 16:49:03 GMT Content-Type=text/html Content-Length=164 target: https://www.storj.io/dcs-satellites FAIL (2025-09-06 16:53:36 UTC) orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n" Duration: 616.708742ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 27 out of 60 is failed (threshold=3) [XX.XXXXX.XX..X..X...XX..XX..X.X.XX....XX.XXX.....X..X....X..] duration: 9.957794961s FAIL (2025-09-06 16:53:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 16:51:40.468819831 (3h23m16.508392951s) [upload=2948]: Starting upload Sep 6 16:51:40.468909502 (3h23m16.508482622s) [upload=2948]: Done starting upload Sep 6 16:51:40.469143449 (3h23m16.508716569s) [upload=2948]: Uploading segments... Sep 6 16:51:40.469186472 (3h23m16.508759592s) [upload=2948]: Got next segment. Inline: false Sep 6 16:51:40.469214385 (3h23m16.508787505s) [upload=2948]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 16:51:40.469219545 (3h23m16.508792665s) [upload=2948]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 16:51:40.720763625 (3h23m16.760336745s) [seg_pos={0 0}, upload=2948]: Begin upload segment... Sep 6 16:51:40.720803491 (3h23m16.760376611s) [seg_pos={0 0}, upload=2948]: Done begin upload segment. Sep 6 16:51:40.720823830 (3h23m16.76039694s) [noise=true , address=45.91.116.9:4263 , piece_id=ACUXJWD4GS7RGXUYPF4RMPTUMVOLYAZ645TINXVGDTGBOWBLBWHA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.720847756 (3h23m16.760420876s) [noise=true , address=77.90.31.19:4843 , piece_id=KBVEXS3RGJE6NG2NDCLI4DJMNIVCIDZGN3XBMJ32QLNW4Q42TMBQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721197264 (3h23m16.760770374s) [noise=true , address=45.91.116.4:4937 , piece_id=ELDRT4NUA5EDMPNLRQDRISVEIM7BGBCCKZBCKSLHUEWMUVZL7AMQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721222492 (3h23m16.760795602s) [noise=true , address=45.91.116.4:4939 , piece_id=6EIHMKXQQEBNFFYHZNJ23GARMCEQQWOJPHWBD4W4USVJKR7JX4NQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721479014 (3h23m16.761052134s) [noise=true , address=45.91.116.6:4974 , piece_id=3QR5JAL7G4OMPA34QZ57MWNKNOWBPSWNDCKROLWOGXMWALO2RPRQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721570098 (3h23m16.761143218s) [noise=true , address=45.91.116.13:4155 , piece_id=4TOH2H4GGC24RIWZF6NMLLLRTBHBGT2BH3IEGZP6CWFKPWCHPL7Q, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721856937 (3h23m16.761430057s) [noise=true , address=77.90.31.14:4662 , piece_id=2HPYC52YRN3FXIAZMKCHMA4AYKZQ5XUIDMXNMAMMOTFUWTCK3VQQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722015730 (3h23m16.76158885s) [noise=true , address=45.91.116.12:4225 , piece_id=KI4LWOQPSV2YZKRXGLWL6GORFJXNPII2RE4TW4U6BJQCU4HRH4NQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722351934 (3h23m16.761925044s) [noise=true , address=45.91.116.5:4297 , piece_id=DU7L2DRLWMLV5OF4FJSVJJ4GIUMWPLQSVDMJVLAIFBE5N3BST6HA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722409674 (3h23m16.761982875s) [noise=true , address=45.91.116.9:4262 , piece_id=LRNRJRYK4LHOILTVMG3RWZ35LTAY24EJALOBF2HWV7AVERS2AGXQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722585891 (3h23m16.762159011s) [noise=true , address=77.90.31.12:4590 , piece_id=5YFKDFUC2CE4KBXYIRC2M55I6TDLJVX3OE4KRLCNNTN3YWOUKFXA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722697465 (3h23m16.762270585s) [noise=true , address=45.91.116.11:4117 , piece_id=5C6V6XTCDPGU2NWZDGSWKREXJBLAJY75XD2XPMDDZU7I5VUPKULQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722810080 (3h23m16.76238319s) [noise=true , address=45.91.116.6:4973 , piece_id=ERHKLODTTAWM6KJJAU76RF2REZMPNEE3UULW2E77DPPT3YEYQFXQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722943706 (3h23m16.762516826s) [noise=true , address=45.91.116.11:4118 , piece_id=XMPTB4CJVYKVSKIQZAPDNJERAGYVDS23ALCULMSZ6MQC65YJML2Q, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.723025482 (3h23m16.762598592s) [noise=true , address=77.90.31.10:4517 , piece_id=UKYCK3PNW73SR2ETCDA6BGJX7EBNRUNPAYTPZSS54BOGUK7KNERA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.723265752 (3h23m16.762838872s) [noise=true , address=45.91.116.10:4083 , piece_id=QNNCGHVWM6ZKDE7VHKH45AXSMSTWR6F4KKNBSRJSRYJQRYLYDCAQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:42.526679200 (3h23m18.56625237s) [upload=2948]: Next returned nil segment Sep 6 16:51:42.526699318 (3h23m18.566272449s) [upload=2948]: Waiting for error group managing segments... Sep 6 16:51:42.823705359 (3h23m18.863278479s) [noise=true , address=45.91.116.6:4973 , piece_id=ERHKLODTTAWM6KJJAU76RF2REZMPNEE3UULW2E77DPPT3YEYQFXQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.840145057 (3h23m18.879718178s) [noise=true , address=45.91.116.10:4083 , piece_id=QNNCGHVWM6ZKDE7VHKH45AXSMSTWR6F4KKNBSRJSRYJQRYLYDCAQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.866065405 (3h23m18.905638525s) [noise=true , address=45.91.116.12:4225 , piece_id=KI4LWOQPSV2YZKRXGLWL6GORFJXNPII2RE4TW4U6BJQCU4HRH4NQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.872872561 (3h23m18.912445681s) [noise=true , address=45.91.116.9:4263 , piece_id=ACUXJWD4GS7RGXUYPF4RMPTUMVOLYAZ645TINXVGDTGBOWBLBWHA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.915160650 (3h23m18.95473377s) [noise=true , address=45.91.116.5:4297 , piece_id=DU7L2DRLWMLV5OF4FJSVJJ4GIUMWPLQSVDMJVLAIFBE5N3BST6HA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.932976680 (3h23m18.9725498s) [noise=true , address=77.90.31.14:4662 , piece_id=2HPYC52YRN3FXIAZMKCHMA4AYKZQ5XUIDMXNMAMMOTFUWTCK3VQQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.960547097 (3h23m19.000120227s) [noise=true , address=45.91.116.11:4117 , piece_id=5C6V6XTCDPGU2NWZDGSWKREXJBLAJY75XD2XPMDDZU7I5VUPKULQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.999827930 (3h23m19.03940105s) [noise=true , address=45.91.116.4:4937 , piece_id=ELDRT4NUA5EDMPNLRQDRISVEIM7BGBCCKZBCKSLHUEWMUVZL7AMQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.032435255 (3h23m19.072008375s) [noise=true , address=45.91.116.9:4262 , piece_id=LRNRJRYK4LHOILTVMG3RWZ35LTAY24EJALOBF2HWV7AVERS2AGXQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.045906972 (3h23m19.085480092s) [noise=true , address=77.90.31.19:4843 , piece_id=KBVEXS3RGJE6NG2NDCLI4DJMNIVCIDZGN3XBMJ32QLNW4Q42TMBQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.050794755 (3h23m19.090367875s) [noise=true , address=45.91.116.13:4155 , piece_id=4TOH2H4GGC24RIWZF6NMLLLRTBHBGT2BH3IEGZP6CWFKPWCHPL7Q, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.056786971 (3h23m19.096360091s) [noise=true , address=77.90.31.12:4590 , piece_id=5YFKDFUC2CE4KBXYIRC2M55I6TDLJVX3OE4KRLCNNTN3YWOUKFXA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.063295826 (3h23m19.102868946s) [noise=true , address=45.91.116.11:4118 , piece_id=XMPTB4CJVYKVSKIQZAPDNJERAGYVDS23ALCULMSZ6MQC65YJML2Q, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.063314001 (3h23m19.102887121s) [seg_pos={0 0}, upload=2948]: Segment reached optimal threshold of 13 pieces. Sep 6 16:51:43.063542228 (3h23m19.103115348s) [noise=true , address=77.90.31.10:4517 , piece_id=UKYCK3PNW73SR2ETCDA6BGJX7EBNRUNPAYTPZSS54BOGUK7KNERA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: write tcp 188.166.225.170:59286->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:59286->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59286->77.90.31.10:4517: use of closed network connection Sep 6 16:51:43.063625216 (3h23m19.103198336s) [noise=true , address=45.91.116.6:4974 , piece_id=3QR5JAL7G4OMPA34QZ57MWNKNOWBPSWNDCKROLWOGXMWALO2RPRQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 6 16:51:43.063699328 (3h23m19.103272448s) [noise=true , address=45.91.116.4:4939 , piece_id=6EIHMKXQQEBNFFYHZNJ23GARMCEQQWOJPHWBD4W4USVJKR7JX4NQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 188.166.225.170:52570->45.91.116.4:4939: use of closed network connection; context canceled Sep 6 16:51:43.063710269 (3h23m19.103283389s) [seg_pos={0 0}, upload=2948]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 16:51:43.063780925 (3h23m19.103354045s) [upload=2948]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 16:51:43.063785022 (3h23m19.103358142s) [upload=2948]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 16:51:43.366135850 (3h23m19.40570897s) [upload=2948]: Done uploading segments... Sep 6 16:51:43.366155037 (3h23m19.405728157s) [upload=2948]: Upload finished. err: <nil> Duration: 9.957827453s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.........X.....] duration: 628.428938ms remote: "tcp:13.233.175.166:443" response headers: X-Cluster-Name=ap-south-1-prod-hosting-red Date=Sat, 06 Sep 2025 16:49:03 GMT Content-Type=text/html Content-Length=164 target: https://www.storj.io/dcs-satellites FAIL (2025-09-06 16:52:36 UTC) orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n" Duration: 628.452082ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 27 out of 60 is failed (threshold=3) [.XX.XXXXX.XX..X..X...XX..XX..X.X.XX....XX.XXX.....X..X....X.] duration: 13.928438991s FAIL (2025-09-06 16:52:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 16:51:40.468819831 (3h23m16.508392951s) [upload=2948]: Starting upload Sep 6 16:51:40.468909502 (3h23m16.508482622s) [upload=2948]: Done starting upload Sep 6 16:51:40.469143449 (3h23m16.508716569s) [upload=2948]: Uploading segments... Sep 6 16:51:40.469186472 (3h23m16.508759592s) [upload=2948]: Got next segment. Inline: false Sep 6 16:51:40.469214385 (3h23m16.508787505s) [upload=2948]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 16:51:40.469219545 (3h23m16.508792665s) [upload=2948]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 16:51:40.720763625 (3h23m16.760336745s) [seg_pos={0 0}, upload=2948]: Begin upload segment... Sep 6 16:51:40.720803491 (3h23m16.760376611s) [seg_pos={0 0}, upload=2948]: Done begin upload segment. Sep 6 16:51:40.720823830 (3h23m16.76039694s) [noise=true , address=45.91.116.9:4263 , piece_id=ACUXJWD4GS7RGXUYPF4RMPTUMVOLYAZ645TINXVGDTGBOWBLBWHA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.720847756 (3h23m16.760420876s) [noise=true , address=77.90.31.19:4843 , piece_id=KBVEXS3RGJE6NG2NDCLI4DJMNIVCIDZGN3XBMJ32QLNW4Q42TMBQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721197264 (3h23m16.760770374s) [noise=true , address=45.91.116.4:4937 , piece_id=ELDRT4NUA5EDMPNLRQDRISVEIM7BGBCCKZBCKSLHUEWMUVZL7AMQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721222492 (3h23m16.760795602s) [noise=true , address=45.91.116.4:4939 , piece_id=6EIHMKXQQEBNFFYHZNJ23GARMCEQQWOJPHWBD4W4USVJKR7JX4NQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721479014 (3h23m16.761052134s) [noise=true , address=45.91.116.6:4974 , piece_id=3QR5JAL7G4OMPA34QZ57MWNKNOWBPSWNDCKROLWOGXMWALO2RPRQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721570098 (3h23m16.761143218s) [noise=true , address=45.91.116.13:4155 , piece_id=4TOH2H4GGC24RIWZF6NMLLLRTBHBGT2BH3IEGZP6CWFKPWCHPL7Q, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721856937 (3h23m16.761430057s) [noise=true , address=77.90.31.14:4662 , piece_id=2HPYC52YRN3FXIAZMKCHMA4AYKZQ5XUIDMXNMAMMOTFUWTCK3VQQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722015730 (3h23m16.76158885s) [noise=true , address=45.91.116.12:4225 , piece_id=KI4LWOQPSV2YZKRXGLWL6GORFJXNPII2RE4TW4U6BJQCU4HRH4NQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722351934 (3h23m16.761925044s) [noise=true , address=45.91.116.5:4297 , piece_id=DU7L2DRLWMLV5OF4FJSVJJ4GIUMWPLQSVDMJVLAIFBE5N3BST6HA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722409674 (3h23m16.761982875s) [noise=true , address=45.91.116.9:4262 , piece_id=LRNRJRYK4LHOILTVMG3RWZ35LTAY24EJALOBF2HWV7AVERS2AGXQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722585891 (3h23m16.762159011s) [noise=true , address=77.90.31.12:4590 , piece_id=5YFKDFUC2CE4KBXYIRC2M55I6TDLJVX3OE4KRLCNNTN3YWOUKFXA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722697465 (3h23m16.762270585s) [noise=true , address=45.91.116.11:4117 , piece_id=5C6V6XTCDPGU2NWZDGSWKREXJBLAJY75XD2XPMDDZU7I5VUPKULQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722810080 (3h23m16.76238319s) [noise=true , address=45.91.116.6:4973 , piece_id=ERHKLODTTAWM6KJJAU76RF2REZMPNEE3UULW2E77DPPT3YEYQFXQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722943706 (3h23m16.762516826s) [noise=true , address=45.91.116.11:4118 , piece_id=XMPTB4CJVYKVSKIQZAPDNJERAGYVDS23ALCULMSZ6MQC65YJML2Q, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.723025482 (3h23m16.762598592s) [noise=true , address=77.90.31.10:4517 , piece_id=UKYCK3PNW73SR2ETCDA6BGJX7EBNRUNPAYTPZSS54BOGUK7KNERA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.723265752 (3h23m16.762838872s) [noise=true , address=45.91.116.10:4083 , piece_id=QNNCGHVWM6ZKDE7VHKH45AXSMSTWR6F4KKNBSRJSRYJQRYLYDCAQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:42.526679200 (3h23m18.56625237s) [upload=2948]: Next returned nil segment Sep 6 16:51:42.526699318 (3h23m18.566272449s) [upload=2948]: Waiting for error group managing segments... Sep 6 16:51:42.823705359 (3h23m18.863278479s) [noise=true , address=45.91.116.6:4973 , piece_id=ERHKLODTTAWM6KJJAU76RF2REZMPNEE3UULW2E77DPPT3YEYQFXQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.840145057 (3h23m18.879718178s) [noise=true , address=45.91.116.10:4083 , piece_id=QNNCGHVWM6ZKDE7VHKH45AXSMSTWR6F4KKNBSRJSRYJQRYLYDCAQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.866065405 (3h23m18.905638525s) [noise=true , address=45.91.116.12:4225 , piece_id=KI4LWOQPSV2YZKRXGLWL6GORFJXNPII2RE4TW4U6BJQCU4HRH4NQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.872872561 (3h23m18.912445681s) [noise=true , address=45.91.116.9:4263 , piece_id=ACUXJWD4GS7RGXUYPF4RMPTUMVOLYAZ645TINXVGDTGBOWBLBWHA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.915160650 (3h23m18.95473377s) [noise=true , address=45.91.116.5:4297 , piece_id=DU7L2DRLWMLV5OF4FJSVJJ4GIUMWPLQSVDMJVLAIFBE5N3BST6HA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.932976680 (3h23m18.9725498s) [noise=true , address=77.90.31.14:4662 , piece_id=2HPYC52YRN3FXIAZMKCHMA4AYKZQ5XUIDMXNMAMMOTFUWTCK3VQQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.960547097 (3h23m19.000120227s) [noise=true , address=45.91.116.11:4117 , piece_id=5C6V6XTCDPGU2NWZDGSWKREXJBLAJY75XD2XPMDDZU7I5VUPKULQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.999827930 (3h23m19.03940105s) [noise=true , address=45.91.116.4:4937 , piece_id=ELDRT4NUA5EDMPNLRQDRISVEIM7BGBCCKZBCKSLHUEWMUVZL7AMQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.032435255 (3h23m19.072008375s) [noise=true , address=45.91.116.9:4262 , piece_id=LRNRJRYK4LHOILTVMG3RWZ35LTAY24EJALOBF2HWV7AVERS2AGXQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.045906972 (3h23m19.085480092s) [noise=true , address=77.90.31.19:4843 , piece_id=KBVEXS3RGJE6NG2NDCLI4DJMNIVCIDZGN3XBMJ32QLNW4Q42TMBQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.050794755 (3h23m19.090367875s) [noise=true , address=45.91.116.13:4155 , piece_id=4TOH2H4GGC24RIWZF6NMLLLRTBHBGT2BH3IEGZP6CWFKPWCHPL7Q, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.056786971 (3h23m19.096360091s) [noise=true , address=77.90.31.12:4590 , piece_id=5YFKDFUC2CE4KBXYIRC2M55I6TDLJVX3OE4KRLCNNTN3YWOUKFXA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.063295826 (3h23m19.102868946s) [noise=true , address=45.91.116.11:4118 , piece_id=XMPTB4CJVYKVSKIQZAPDNJERAGYVDS23ALCULMSZ6MQC65YJML2Q, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.063314001 (3h23m19.102887121s) [seg_pos={0 0}, upload=2948]: Segment reached optimal threshold of 13 pieces. Sep 6 16:51:43.063542228 (3h23m19.103115348s) [noise=true , address=77.90.31.10:4517 , piece_id=UKYCK3PNW73SR2ETCDA6BGJX7EBNRUNPAYTPZSS54BOGUK7KNERA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: write tcp 188.166.225.170:59286->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:59286->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59286->77.90.31.10:4517: use of closed network connection Sep 6 16:51:43.063625216 (3h23m19.103198336s) [noise=true , address=45.91.116.6:4974 , piece_id=3QR5JAL7G4OMPA34QZ57MWNKNOWBPSWNDCKROLWOGXMWALO2RPRQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 6 16:51:43.063699328 (3h23m19.103272448s) [noise=true , address=45.91.116.4:4939 , piece_id=6EIHMKXQQEBNFFYHZNJ23GARMCEQQWOJPHWBD4W4USVJKR7JX4NQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 188.166.225.170:52570->45.91.116.4:4939: use of closed network connection; context canceled Sep 6 16:51:43.063710269 (3h23m19.103283389s) [seg_pos={0 0}, upload=2948]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 16:51:43.063780925 (3h23m19.103354045s) [upload=2948]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 16:51:43.063785022 (3h23m19.103358142s) [upload=2948]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 16:51:43.366135850 (3h23m19.40570897s) [upload=2948]: Done uploading segments... Sep 6 16:51:43.366155037 (3h23m19.405728157s) [upload=2948]: Upload finished. err: <nil> Duration: 13.928466143s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..........X....] duration: 705.450288ms remote: "tcp:13.233.175.166:443" response headers: X-Cluster-Name=ap-south-1-prod-hosting-red Date=Sat, 06 Sep 2025 16:49:03 GMT Content-Type=text/html Content-Length=164 target: https://www.storj.io/dcs-satellites FAIL (2025-09-06 16:52:19 UTC) orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n" Duration: 705.473023ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 28 out of 60 is failed (threshold=3) [X.XX.XXXXX.XX..X..X...XX..XX..X.X.XX....XX.XXX.....X..X....X] duration: 30.002419114s FAIL (2025-09-06 16:52:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 16:51:40.468819831 (3h23m16.508392951s) [upload=2948]: Starting upload Sep 6 16:51:40.468909502 (3h23m16.508482622s) [upload=2948]: Done starting upload Sep 6 16:51:40.469143449 (3h23m16.508716569s) [upload=2948]: Uploading segments... Sep 6 16:51:40.469186472 (3h23m16.508759592s) [upload=2948]: Got next segment. Inline: false Sep 6 16:51:40.469214385 (3h23m16.508787505s) [upload=2948]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 16:51:40.469219545 (3h23m16.508792665s) [upload=2948]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 16:51:40.720763625 (3h23m16.760336745s) [seg_pos={0 0}, upload=2948]: Begin upload segment... Sep 6 16:51:40.720803491 (3h23m16.760376611s) [seg_pos={0 0}, upload=2948]: Done begin upload segment. Sep 6 16:51:40.720823830 (3h23m16.76039694s) [noise=true , address=45.91.116.9:4263 , piece_id=ACUXJWD4GS7RGXUYPF4RMPTUMVOLYAZ645TINXVGDTGBOWBLBWHA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.720847756 (3h23m16.760420876s) [noise=true , address=77.90.31.19:4843 , piece_id=KBVEXS3RGJE6NG2NDCLI4DJMNIVCIDZGN3XBMJ32QLNW4Q42TMBQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721197264 (3h23m16.760770374s) [noise=true , address=45.91.116.4:4937 , piece_id=ELDRT4NUA5EDMPNLRQDRISVEIM7BGBCCKZBCKSLHUEWMUVZL7AMQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721222492 (3h23m16.760795602s) [noise=true , address=45.91.116.4:4939 , piece_id=6EIHMKXQQEBNFFYHZNJ23GARMCEQQWOJPHWBD4W4USVJKR7JX4NQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721479014 (3h23m16.761052134s) [noise=true , address=45.91.116.6:4974 , piece_id=3QR5JAL7G4OMPA34QZ57MWNKNOWBPSWNDCKROLWOGXMWALO2RPRQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721570098 (3h23m16.761143218s) [noise=true , address=45.91.116.13:4155 , piece_id=4TOH2H4GGC24RIWZF6NMLLLRTBHBGT2BH3IEGZP6CWFKPWCHPL7Q, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.721856937 (3h23m16.761430057s) [noise=true , address=77.90.31.14:4662 , piece_id=2HPYC52YRN3FXIAZMKCHMA4AYKZQ5XUIDMXNMAMMOTFUWTCK3VQQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722015730 (3h23m16.76158885s) [noise=true , address=45.91.116.12:4225 , piece_id=KI4LWOQPSV2YZKRXGLWL6GORFJXNPII2RE4TW4U6BJQCU4HRH4NQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722351934 (3h23m16.761925044s) [noise=true , address=45.91.116.5:4297 , piece_id=DU7L2DRLWMLV5OF4FJSVJJ4GIUMWPLQSVDMJVLAIFBE5N3BST6HA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722409674 (3h23m16.761982875s) [noise=true , address=45.91.116.9:4262 , piece_id=LRNRJRYK4LHOILTVMG3RWZ35LTAY24EJALOBF2HWV7AVERS2AGXQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722585891 (3h23m16.762159011s) [noise=true , address=77.90.31.12:4590 , piece_id=5YFKDFUC2CE4KBXYIRC2M55I6TDLJVX3OE4KRLCNNTN3YWOUKFXA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722697465 (3h23m16.762270585s) [noise=true , address=45.91.116.11:4117 , piece_id=5C6V6XTCDPGU2NWZDGSWKREXJBLAJY75XD2XPMDDZU7I5VUPKULQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722810080 (3h23m16.76238319s) [noise=true , address=45.91.116.6:4973 , piece_id=ERHKLODTTAWM6KJJAU76RF2REZMPNEE3UULW2E77DPPT3YEYQFXQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.722943706 (3h23m16.762516826s) [noise=true , address=45.91.116.11:4118 , piece_id=XMPTB4CJVYKVSKIQZAPDNJERAGYVDS23ALCULMSZ6MQC65YJML2Q, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.723025482 (3h23m16.762598592s) [noise=true , address=77.90.31.10:4517 , piece_id=UKYCK3PNW73SR2ETCDA6BGJX7EBNRUNPAYTPZSS54BOGUK7KNERA, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:40.723265752 (3h23m16.762838872s) [noise=true , address=45.91.116.10:4083 , piece_id=QNNCGHVWM6ZKDE7VHKH45AXSMSTWR6F4KKNBSRJSRYJQRYLYDCAQ, seg_pos={0 0}, upload=2948]: Uploading piece... Sep 6 16:51:42.526679200 (3h23m18.56625237s) [upload=2948]: Next returned nil segment Sep 6 16:51:42.526699318 (3h23m18.566272449s) [upload=2948]: Waiting for error group managing segments... Sep 6 16:51:42.823705359 (3h23m18.863278479s) [noise=true , address=45.91.116.6:4973 , piece_id=ERHKLODTTAWM6KJJAU76RF2REZMPNEE3UULW2E77DPPT3YEYQFXQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.840145057 (3h23m18.879718178s) [noise=true , address=45.91.116.10:4083 , piece_id=QNNCGHVWM6ZKDE7VHKH45AXSMSTWR6F4KKNBSRJSRYJQRYLYDCAQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.866065405 (3h23m18.905638525s) [noise=true , address=45.91.116.12:4225 , piece_id=KI4LWOQPSV2YZKRXGLWL6GORFJXNPII2RE4TW4U6BJQCU4HRH4NQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.872872561 (3h23m18.912445681s) [noise=true , address=45.91.116.9:4263 , piece_id=ACUXJWD4GS7RGXUYPF4RMPTUMVOLYAZ645TINXVGDTGBOWBLBWHA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.915160650 (3h23m18.95473377s) [noise=true , address=45.91.116.5:4297 , piece_id=DU7L2DRLWMLV5OF4FJSVJJ4GIUMWPLQSVDMJVLAIFBE5N3BST6HA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.932976680 (3h23m18.9725498s) [noise=true , address=77.90.31.14:4662 , piece_id=2HPYC52YRN3FXIAZMKCHMA4AYKZQ5XUIDMXNMAMMOTFUWTCK3VQQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.960547097 (3h23m19.000120227s) [noise=true , address=45.91.116.11:4117 , piece_id=5C6V6XTCDPGU2NWZDGSWKREXJBLAJY75XD2XPMDDZU7I5VUPKULQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:42.999827930 (3h23m19.03940105s) [noise=true , address=45.91.116.4:4937 , piece_id=ELDRT4NUA5EDMPNLRQDRISVEIM7BGBCCKZBCKSLHUEWMUVZL7AMQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.032435255 (3h23m19.072008375s) [noise=true , address=45.91.116.9:4262 , piece_id=LRNRJRYK4LHOILTVMG3RWZ35LTAY24EJALOBF2HWV7AVERS2AGXQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.045906972 (3h23m19.085480092s) [noise=true , address=77.90.31.19:4843 , piece_id=KBVEXS3RGJE6NG2NDCLI4DJMNIVCIDZGN3XBMJ32QLNW4Q42TMBQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.050794755 (3h23m19.090367875s) [noise=true , address=45.91.116.13:4155 , piece_id=4TOH2H4GGC24RIWZF6NMLLLRTBHBGT2BH3IEGZP6CWFKPWCHPL7Q, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.056786971 (3h23m19.096360091s) [noise=true , address=77.90.31.12:4590 , piece_id=5YFKDFUC2CE4KBXYIRC2M55I6TDLJVX3OE4KRLCNNTN3YWOUKFXA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.063295826 (3h23m19.102868946s) [noise=true , address=45.91.116.11:4118 , piece_id=XMPTB4CJVYKVSKIQZAPDNJERAGYVDS23ALCULMSZ6MQC65YJML2Q, seg_pos={0 0}, upload=2948]: Done uploading piece. err: <nil> Sep 6 16:51:43.063314001 (3h23m19.102887121s) [seg_pos={0 0}, upload=2948]: Segment reached optimal threshold of 13 pieces. Sep 6 16:51:43.063542228 (3h23m19.103115348s) [noise=true , address=77.90.31.10:4517 , piece_id=UKYCK3PNW73SR2ETCDA6BGJX7EBNRUNPAYTPZSS54BOGUK7KNERA, seg_pos={0 0}, upload=2948]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: write tcp 188.166.225.170:59286->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:59286->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59286->77.90.31.10:4517: use of closed network connection Sep 6 16:51:43.063625216 (3h23m19.103198336s) [noise=true , address=45.91.116.6:4974 , piece_id=3QR5JAL7G4OMPA34QZ57MWNKNOWBPSWNDCKROLWOGXMWALO2RPRQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Sep 6 16:51:43.063699328 (3h23m19.103272448s) [noise=true , address=45.91.116.4:4939 , piece_id=6EIHMKXQQEBNFFYHZNJ23GARMCEQQWOJPHWBD4W4USVJKR7JX4NQ, seg_pos={0 0}, upload=2948]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 188.166.225.170:52570->45.91.116.4:4939: use of closed network connection; context canceled Sep 6 16:51:43.063710269 (3h23m19.103283389s) [seg_pos={0 0}, upload=2948]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 16:51:43.063780925 (3h23m19.103354045s) [upload=2948]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 16:51:43.063785022 (3h23m19.103358142s) [upload=2948]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 16:51:43.366135850 (3h23m19.40570897s) [upload=2948]: Done uploading segments... Sep 6 16:51:43.366155037 (3h23m19.405728157s) [upload=2948]: Upload finished. err: <nil> Duration: 30.002447968s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [...........X...] duration: 598.780407ms remote: "tcp:13.233.175.166:443" response headers: X-Cluster-Name=ap-south-1-prod-hosting-red Date=Sat, 06 Sep 2025 16:49:03 GMT Content-Type=text/html Content-Length=164 target: https://www.storj.io/dcs-satellites FAIL (2025-09-06 16:51:21 UTC) orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n" Duration: 598.794944ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [............X..] duration: 751.995473ms remote: "tcp:13.233.175.166:443" response headers: X-Cluster-Name=ap-south-1-prod-hosting-red Date=Sat, 06 Sep 2025 16:49:03 GMT Content-Type=text/html Content-Length=164 target: https://www.storj.io/dcs-satellites FAIL (2025-09-06 16:51:00 UTC) orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n" Duration: 752.024519ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 27 out of 60 is failed (threshold=3) [.X.XX.XXXXX.XX..X..X...XX..XX..X.X.XX....XX.XXX.....X..X....] duration: 9.520987097s FAIL (2025-09-06 16:50:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 16:47:57.508531272 (3h19m33.548104392s) [upload=2898]: Starting upload Sep 6 16:47:57.508588271 (3h19m33.548161391s) [upload=2898]: Done starting upload Sep 6 16:47:57.508658075 (3h19m33.548231205s) [upload=2898]: Uploading segments... Sep 6 16:47:57.508772323 (3h19m33.548345433s) [upload=2898]: Got next segment. Inline: false Sep 6 16:47:57.508802401 (3h19m33.548375521s) [upload=2898]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 16:47:57.508806418 (3h19m33.548379538s) [upload=2898]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 16:47:57.755764132 (3h19m33.795337242s) [seg_pos={0 0}, upload=2898]: Begin upload segment... Sep 6 16:47:57.755873381 (3h19m33.795446581s) [seg_pos={0 0}, upload=2898]: Done begin upload segment. Sep 6 16:47:57.755888170 (3h19m33.79546129s) [noise=true , address=45.91.116.4:4939 , piece_id=SPXXNVRUEYTO6SQBGMRJWBD3KDGD24FZYBHBL7N6LEHECAQX2IPA, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.755904360 (3h19m33.79547748s) [noise=true , address=77.90.31.14:4661 , piece_id=CXBW2QDND7JJRS4CVRDHJG5FZPMO6WTEHAXOAGF5GVPI7CUGUTLA, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.756222870 (3h19m33.79579599s) [noise=true , address=77.90.31.18:4806 , piece_id=QZRI3JKFAI4YNZQ5F4RFVMWJHKK2WDEMVNZGXHQFUECYRSN4I4HA, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.756242628 (3h19m33.795815748s) [noise=true , address=45.91.116.11:4118 , piece_id=5GNKZLFRSP36JGDOMSYJYF5OZMNOXUVH5TKDRFRNLKK25CRHMB7A, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.756513826 (3h19m33.796086946s) [noise=true , address=45.91.116.6:4974 , piece_id=6X2JYEFK7YZVR5BTJO6BJ6TCFBP326ORBGGLLRASGXVVOS57ALMQ, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.756635549 (3h19m33.796208659s) [noise=true , address=45.91.116.11:4119 , piece_id=QX2DHX4AL4CI6RGLRBTAVLXWFZ2ZFTGUJJGAZIOCZ2ZFKQOCEANQ, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.756775607 (3h19m33.796348717s) [noise=true , address=77.90.31.14:4663 , piece_id=JT2M2ZSAL522G7QTUSSLCBPFDMFPXIW35LMP2ICUYKSEJLMV4O2Q, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.756960321 (3h19m33.796533441s) [noise=true , address=77.90.31.10:4520 , piece_id=3XOQQDPIYSDSCRUZRRMZ24JWNTLBYA4R2YK7IARF6L5G6T4DQSLA, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.757041326 (3h19m33.796614446s) [noise=true , address=45.91.116.10:4082 , piece_id=IAEAOCAVNPDWYO2N2VWZDIRQHCMKHIBNXAMAENO3ZYQKWI76ECAQ, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.757272368 (3h19m33.796845488s) [noise=true , address=45.91.116.4:4940 , piece_id=LABNRF2O3S34D3FFHWJP7G4OR7BXJZPBU5BK2S4LKCZ3SHKOJPYQ, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.757314328 (3h19m33.796887448s) [noise=true , address=45.91.116.6:4973 , piece_id=GOQVRGQZIYSXOCRUNV5JDQ3X3U2GNREVZYYNGHNGDLH7EO75JEBA, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.757664819 (3h19m33.797237939s) [noise=true , address=45.91.116.7:4011 , piece_id=HPY3HV7FBSRK5F6ZLS25WXM34W6NXIQ7F5QQ2BWBE3LT5MAVAZGQ, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.757947400 (3h19m33.79752051s) [noise=true , address=45.91.116.8:4045 , piece_id=I2UIIVLERLJSBMEKJSSCZRXKAUB5YSNKN5OQ4KSEOBA4O7IDWMJQ, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.758170607 (3h19m33.797743727s) [noise=true , address=45.91.116.13:4155 , piece_id=OHONMM5OMTOXGNXBHLKEMKTYBUOCYG22UC4GSGSHECDTORWGYPYA, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.758446114 (3h19m33.798019234s) [noise=true , address=77.90.31.16:4734 , piece_id=JJLKMX5VN6VSSGMYL7NPWC3URYFK3YE2D3QXCQCKCNB4HPAVUYUQ, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.758864875 (3h19m33.798437995s) [noise=true , address=45.91.116.5:4297 , piece_id=GV7VXXFYB6YANXDQHIFII32XHTGJACQPP3BXVDZX2KNGBDPHEAKA, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:59.591954020 (3h19m35.63152714s) [upload=2898]: Next returned nil segment Sep 6 16:47:59.591970833 (3h19m35.631543953s) [upload=2898]: Waiting for error group managing segments... Sep 6 16:47:59.888947738 (3h19m35.928520858s) [noise=true , address=45.91.116.4:4940 , piece_id=LABNRF2O3S34D3FFHWJP7G4OR7BXJZPBU5BK2S4LKCZ3SHKOJPYQ, seg_pos={0 0}, upload=2898]: Done uploading piece. err: <nil> Sep 6 16:47:59.925607617 (3h19m35.965180737s) [noise=true , address=45.91.116.6:4973 , piece_id=GOQVRGQZIYSXOCRUNV5JDQ3X3U2GNREVZYYNGHNGDLH7EO75JEBA, seg_pos={0 0}, upload=2898]: Done uploading piece. err: <nil> Sep 6 16:47:59.943738240 (3h19m35.98331136s) [noise=true , address=45.91.116.4:4939 , piece_id=SPXXNVRUEYTO6SQBGMRJWBD3KDGD24FZYBHBL7N6LEHECAQX2IPA, seg_pos={0 0}, upload=2898]: Done uploading piece. err: <nil> Sep 6 16:47:59.944472264 (3h19m35.984045384s) [noise=true , address=45.91.116.11:4118 , piece_id=5GNKZLFRSP36JGDOMSYJYF5OZMNOXUVH5TKDRFRNLKK25CRHMB7A, seg_pos={0 0}, upload=2898]: Done uploading piece. err: <nil> Sep 6 16:47:59.963769570 (3h19m36.00334269s) [noise=true , address=45.91.116.13:4155 , piece_id=OHONMM5OMTOXGNXBHLKEMKTYBUOCYG22UC4GSGSHECDTORWGYPYA, seg_pos={0 0}, upload=2898]: Done uploading piece. err: <nil> Sep 6 16:47:59.971966224 (3h19m36.011539344s) [noise=true , address=77.90.31.14:4661 , piece_id=CXBW2QDND7JJRS4CVRDHJG5FZPMO6WTEHAXOAGF5GVPI7CUGUTLA, seg_pos={0 0}, upload=2898]: Done uploading piece. err: <nil> Sep 6 16:47:59.972061055 (3h19m36.011634175s) [noise=true , address=45.91.116.6:4974 , piece_id=6X2JYEFK7YZVR5BTJO6BJ6TCFBP326ORBGGLLRASGXVVOS57ALMQ, seg_pos={0 0}, upload=2898]: Done uploading piece. err: <nil> Sep 6 16:48:00.012351391 (3h19m36.051924541s) [noise=true , address=45.91.116.5:4297 , piece_id=GV7VXXFYB6YANXDQHIFII32XHTGJACQPP3BXVDZX2KNGBDPHEAKA, seg_pos={0 0}, upload=2898]: Done uploading piece. err: <nil> Sep 6 16:48:00.066142711 (3h19m36.105715841s) [noise=true , address=45.91.116.8:4045 , piece_id=I2UIIVLERLJSBMEKJSSCZRXKAUB5YSNKN5OQ4KSEOBA4O7IDWMJQ, seg_pos={0 0}, upload=2898]: Done uploading piece. err: <nil> Sep 6 16:48:00.086714004 (3h19m36.126287124s) [noise=true , address=45.91.116.10:4082 , piece_id=IAEAOCAVNPDWYO2N2VWZDIRQHCMKHIBNXAMAENO3ZYQKWI76ECAQ, seg_pos={0 0}, upload=2898]: Done uploading piece. err: <nil> Sep 6 16:48:00.088672672 (3h19m36.128245792s) [noise=true , address=45.91.116.11:4119 , piece_id=QX2DHX4AL4CI6RGLRBTAVLXWFZ2ZFTGUJJGAZIOCZ2ZFKQOCEANQ, seg_pos={0 0}, upload=2898]: Done uploading piece. err: <nil> Sep 6 16:48:00.115185644 (3h19m36.154758794s) [noise=true , address=45.91.116.7:4011 , piece_id=HPY3HV7FBSRK5F6ZLS25WXM34W6NXIQ7F5QQ2BWBE3LT5MAVAZGQ, seg_pos={0 0}, upload=2898]: Done uploading piece. err: <nil> Sep 6 16:48:00.159549154 (3h19m36.199122274s) [noise=true , address=77.90.31.14:4663 , piece_id=JT2M2ZSAL522G7QTUSSLCBPFDMFPXIW35LMP2ICUYKSEJLMV4O2Q, seg_pos={0 0}, upload=2898]: Done uploading piece. err: <nil> Sep 6 16:48:00.159575936 (3h19m36.199149056s) [seg_pos={0 0}, upload=2898]: Segment reached optimal threshold of 13 pieces. Sep 6 16:48:00.159750149 (3h19m36.199323269s) [noise=true , address=77.90.31.10:4520 , piece_id=3XOQQDPIYSDSCRUZRRMZ24JWNTLBYA4R2YK7IARF6L5G6T4DQSLA, seg_pos={0 0}, upload=2898]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Sep 6 16:48:00.159871069 (3h19m36.19944419s) [noise=true , address=77.90.31.16:4734 , piece_id=JJLKMX5VN6VSSGMYL7NPWC3URYFK3YE2D3QXCQCKCNB4HPAVUYUQ, seg_pos={0 0}, upload=2898]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: write tcp 188.166.225.170:55152->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:55152->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55152->77.90.31.16:4734: use of closed network connection Sep 6 16:48:00.159910526 (3h19m36.199483646s) [noise=true , address=77.90.31.18:4806 , piece_id=QZRI3JKFAI4YNZQ5F4RFVMWJHKK2WDEMVNZGXHQFUECYRSN4I4HA, seg_pos={0 0}, upload=2898]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: write tcp 188.166.225.170:60398->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 16:48:00.159928940 (3h19m36.19950206s) [seg_pos={0 0}, upload=2898]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 16:48:00.159989397 (3h19m36.199562517s) [upload=2898]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 16:48:00.159993154 (3h19m36.199566274s) [upload=2898]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 16:48:00.557217011 (3h19m36.596790151s) [upload=2898]: Done uploading segments... Sep 6 16:48:00.557238753 (3h19m36.596811883s) [upload=2898]: Upload finished. err: <nil> Duration: 9.521035178s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 28 out of 60 is failed (threshold=3) [X.X.XX.XXXXX.XX..X..X...XX..XX..X.X.XX....XX.XXX.....X..X...] duration: 9.992033291s FAIL (2025-09-06 16:50:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 16:47:57.508531272 (3h19m33.548104392s) [upload=2898]: Starting upload Sep 6 16:47:57.508588271 (3h19m33.548161391s) [upload=2898]: Done starting upload Sep 6 16:47:57.508658075 (3h19m33.548231205s) [upload=2898]: Uploading segments... Sep 6 16:47:57.508772323 (3h19m33.548345433s) [upload=2898]: Got next segment. Inline: false Sep 6 16:47:57.508802401 (3h19m33.548375521s) [upload=2898]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 16:47:57.508806418 (3h19m33.548379538s) [upload=2898]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 16:47:57.755764132 (3h19m33.795337242s) [seg_pos={0 0}, upload=2898]: Begin upload segment... Sep 6 16:47:57.755873381 (3h19m33.795446581s) [seg_pos={0 0}, upload=2898]: Done begin upload segment. Sep 6 16:47:57.755888170 (3h19m33.79546129s) [noise=true , address=45.91.116.4:4939 , piece_id=SPXXNVRUEYTO6SQBGMRJWBD3KDGD24FZYBHBL7N6LEHECAQX2IPA, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.755904360 (3h19m33.79547748s) [noise=true , address=77.90.31.14:4661 , piece_id=CXBW2QDND7JJRS4CVRDHJG5FZPMO6WTEHAXOAGF5GVPI7CUGUTLA, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.756222870 (3h19m33.79579599s) [noise=true , address=77.90.31.18:4806 , piece_id=QZRI3JKFAI4YNZQ5F4RFVMWJHKK2WDEMVNZGXHQFUECYRSN4I4HA, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.756242628 (3h19m33.795815748s) [noise=true , address=45.91.116.11:4118 , piece_id=5GNKZLFRSP36JGDOMSYJYF5OZMNOXUVH5TKDRFRNLKK25CRHMB7A, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.756513826 (3h19m33.796086946s) [noise=true , address=45.91.116.6:4974 , piece_id=6X2JYEFK7YZVR5BTJO6BJ6TCFBP326ORBGGLLRASGXVVOS57ALMQ, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.756635549 (3h19m33.796208659s) [noise=true , address=45.91.116.11:4119 , piece_id=QX2DHX4AL4CI6RGLRBTAVLXWFZ2ZFTGUJJGAZIOCZ2ZFKQOCEANQ, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.756775607 (3h19m33.796348717s) [noise=true , address=77.90.31.14:4663 , piece_id=JT2M2ZSAL522G7QTUSSLCBPFDMFPXIW35LMP2ICUYKSEJLMV4O2Q, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.756960321 (3h19m33.796533441s) [noise=true , address=77.90.31.10:4520 , piece_id=3XOQQDPIYSDSCRUZRRMZ24JWNTLBYA4R2YK7IARF6L5G6T4DQSLA, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.757041326 (3h19m33.796614446s) [noise=true , address=45.91.116.10:4082 , piece_id=IAEAOCAVNPDWYO2N2VWZDIRQHCMKHIBNXAMAENO3ZYQKWI76ECAQ, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.757272368 (3h19m33.796845488s) [noise=true , address=45.91.116.4:4940 , piece_id=LABNRF2O3S34D3FFHWJP7G4OR7BXJZPBU5BK2S4LKCZ3SHKOJPYQ, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.757314328 (3h19m33.796887448s) [noise=true , address=45.91.116.6:4973 , piece_id=GOQVRGQZIYSXOCRUNV5JDQ3X3U2GNREVZYYNGHNGDLH7EO75JEBA, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.757664819 (3h19m33.797237939s) [noise=true , address=45.91.116.7:4011 , piece_id=HPY3HV7FBSRK5F6ZLS25WXM34W6NXIQ7F5QQ2BWBE3LT5MAVAZGQ, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.757947400 (3h19m33.79752051s) [noise=true , address=45.91.116.8:4045 , piece_id=I2UIIVLERLJSBMEKJSSCZRXKAUB5YSNKN5OQ4KSEOBA4O7IDWMJQ, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.758170607 (3h19m33.797743727s) [noise=true , address=45.91.116.13:4155 , piece_id=OHONMM5OMTOXGNXBHLKEMKTYBUOCYG22UC4GSGSHECDTORWGYPYA, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.758446114 (3h19m33.798019234s) [noise=true , address=77.90.31.16:4734 , piece_id=JJLKMX5VN6VSSGMYL7NPWC3URYFK3YE2D3QXCQCKCNB4HPAVUYUQ, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:57.758864875 (3h19m33.798437995s) [noise=true , address=45.91.116.5:4297 , piece_id=GV7VXXFYB6YANXDQHIFII32XHTGJACQPP3BXVDZX2KNGBDPHEAKA, seg_pos={0 0}, upload=2898]: Uploading piece... Sep 6 16:47:59.591954020 (3h19m35.63152714s) [upload=2898]: Next returned nil segment Sep 6 16:47:59.591970833 (3h19m35.631543953s) [upload=2898]: Waiting for error group managing segments... Sep 6 16:47:59.888947738 (3h19m35.928520858s) [noise=true , address=45.91.116.4:4940 , piece_id=LABNRF2O3S34D3FFHWJP7G4OR7BXJZPBU5BK2S4LKCZ3SHKOJPYQ, seg_pos={0 0}, upload=2898]: Done uploading piece. err: <nil> Sep 6 16:47:59.925607617 (3h19m35.965180737s) [noise=true , address=45.91.116.6:4973 , piece_id=GOQVRGQZIYSXOCRUNV5JDQ3X3U2GNREVZYYNGHNGDLH7EO75JEBA, seg_pos={0 0}, upload=2898]: Done uploading piece. err: <nil> Sep 6 16:47:59.943738240 (3h19m35.98331136s) [noise=true , address=45.91.116.4:4939 , piece_id=SPXXNVRUEYTO6SQBGMRJWBD3KDGD24FZYBHBL7N6LEHECAQX2IPA, seg_pos={0 0}, upload=2898]: Done uploading piece. err: <nil> Sep 6 16:47:59.944472264 (3h19m35.984045384s) [noise=true , address=45.91.116.11:4118 , piece_id=5GNKZLFRSP36JGDOMSYJYF5OZMNOXUVH5TKDRFRNLKK25CRHMB7A, seg_pos={0 0}, upload=2898]: Done uploading piece. err: <nil> Sep 6 16:47:59.963769570 (3h19m36.00334269s) [noise=true , address=45.91.116.13:4155 , piece_id=OHONMM5OMTOXGNXBHLKEMKTYBUOCYG22UC4GSGSHECDTORWGYPYA, seg_pos={0 0}, upload=2898]: Done uploading piece. err: <nil> Sep 6 16:47:59.971966224 (3h19m36.011539344s) [noise=true , address=77.90.31.14:4661 , piece_id=CXBW2QDND7JJRS4CVRDHJG5FZPMO6WTEHAXOAGF5GVPI7CUGUTLA, seg_pos={0 0}, upload=2898]: Done uploading piece. err: <nil> Sep 6 16:47:59.972061055 (3h19m36.011634175s) [noise=true , address=45.91.116.6:4974 , piece_id=6X2JYEFK7YZVR5BTJO6BJ6TCFBP326ORBGGLLRASGXVVOS57ALMQ, seg_pos={0 0}, upload=2898]: Done uploading piece. err: <nil> Sep 6 16:48:00.012351391 (3h19m36.051924541s) [noise=true , address=45.91.116.5:4297 , piece_id=GV7VXXFYB6YANXDQHIFII32XHTGJACQPP3BXVDZX2KNGBDPHEAKA, seg_pos={0 0}, upload=2898]: Done uploading piece. err: <nil> Sep 6 16:48:00.066142711 (3h19m36.105715841s) [noise=true , address=45.91.116.8:4045 , piece_id=I2UIIVLERLJSBMEKJSSCZRXKAUB5YSNKN5OQ4KSEOBA4O7IDWMJQ, seg_pos={0 0}, upload=2898]: Done uploading piece. err: <nil> Sep 6 16:48:00.086714004 (3h19m36.126287124s) [noise=true , address=45.91.116.10:4082 , piece_id=IAEAOCAVNPDWYO2N2VWZDIRQHCMKHIBNXAMAENO3ZYQKWI76ECAQ, seg_pos={0 0}, upload=2898]: Done uploading piece. err: <nil> Sep 6 16:48:00.088672672 (3h19m36.128245792s) [noise=true , address=45.91.116.11:4119 , piece_id=QX2DHX4AL4CI6RGLRBTAVLXWFZ2ZFTGUJJGAZIOCZ2ZFKQOCEANQ, seg_pos={0 0}, upload=2898]: Done uploading piece. err: <nil> Sep 6 16:48:00.115185644 (3h19m36.154758794s) [noise=true , address=45.91.116.7:4011 , piece_id=HPY3HV7FBSRK5F6ZLS25WXM34W6NXIQ7F5QQ2BWBE3LT5MAVAZGQ, seg_pos={0 0}, upload=2898]: Done uploading piece. err: <nil> Sep 6 16:48:00.159549154 (3h19m36.199122274s) [noise=true , address=77.90.31.14:4663 , piece_id=JT2M2ZSAL522G7QTUSSLCBPFDMFPXIW35LMP2ICUYKSEJLMV4O2Q, seg_pos={0 0}, upload=2898]: Done uploading piece. err: <nil> Sep 6 16:48:00.159575936 (3h19m36.199149056s) [seg_pos={0 0}, upload=2898]: Segment reached optimal threshold of 13 pieces. Sep 6 16:48:00.159750149 (3h19m36.199323269s) [noise=true , address=77.90.31.10:4520 , piece_id=3XOQQDPIYSDSCRUZRRMZ24JWNTLBYA4R2YK7IARF6L5G6T4DQSLA, seg_pos={0 0}, upload=2898]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Sep 6 16:48:00.159871069 (3h19m36.19944419s) [noise=true , address=77.90.31.16:4734 , piece_id=JJLKMX5VN6VSSGMYL7NPWC3URYFK3YE2D3QXCQCKCNB4HPAVUYUQ, seg_pos={0 0}, upload=2898]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: write tcp 188.166.225.170:55152->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:55152->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55152->77.90.31.16:4734: use of closed network connection Sep 6 16:48:00.159910526 (3h19m36.199483646s) [noise=true , address=77.90.31.18:4806 , piece_id=QZRI3JKFAI4YNZQ5F4RFVMWJHKK2WDEMVNZGXHQFUECYRSN4I4HA, seg_pos={0 0}, upload=2898]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: write tcp 188.166.225.170:60398->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 16:48:00.159928940 (3h19m36.19950206s) [seg_pos={0 0}, upload=2898]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 16:48:00.159989397 (3h19m36.199562517s) [upload=2898]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 16:48:00.159993154 (3h19m36.199566274s) [upload=2898]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 16:48:00.557217011 (3h19m36.596790151s) [upload=2898]: Done uploading segments... Sep 6 16:48:00.557238753 (3h19m36.596811883s) [upload=2898]: Upload finished. err: <nil> Duration: 9.992072716s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)