Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 17.810681717s FAIL (2025-09-19 14:12:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 17.810731041s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 7.428248166s FAIL (2025-09-19 14:12:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 7.428289915s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 13.720089068s FAIL (2025-09-19 14:10:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 13.720138142s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 8.15327318s FAIL (2025-09-19 14:10:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 8.153305911s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 18.711034526s FAIL (2025-09-19 14:09:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 18.711069002s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 7.25532496s FAIL (2025-09-19 14:08:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 7.255355388s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 14.49840252s FAIL (2025-09-19 14:07:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 14.498436976s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 11.616304133s FAIL (2025-09-19 14:06:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 11.616341414s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 14.506966967s FAIL (2025-09-19 14:06:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 14.506996101s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 8.296718316s FAIL (2025-09-19 14:04:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 8.29676284s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 9.748689229s FAIL (2025-09-19 14:04:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 9.748725729s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 23.321502202s FAIL (2025-09-19 14:03:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 23.321626378s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 10.882446191s FAIL (2025-09-19 14:03:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 10.882472741s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 8.222309215s FAIL (2025-09-19 14:02:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 8.222344091s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 18.364297009s FAIL (2025-09-19 14:01:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 18.364321055s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 20.748819276s FAIL (2025-09-19 14:00:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 20.748856006s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 11.586009205s FAIL (2025-09-19 14:00:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 11.586185089s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 12.763346827s FAIL (2025-09-19 13:59:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 12.763391382s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 12.01848204s FAIL (2025-09-19 13:58:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 12.018510505s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 18.200194857s FAIL (2025-09-19 13:57:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 18.200220897s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 19.625509428s FAIL (2025-09-19 13:57:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 19.625539676s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 23.115441686s FAIL (2025-09-19 13:56:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 23.115468837s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 13.18512384s FAIL (2025-09-19 13:56:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 13.18516045s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 12.710686169s FAIL (2025-09-19 13:55:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 12.710721295s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 6.904456096s FAIL (2025-09-19 13:54:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 6.904489859s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 12.978835019s FAIL (2025-09-19 13:54:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 12.978865857s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 9.90227053s FAIL (2025-09-19 13:53:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 9.90230749s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 10.573490384s FAIL (2025-09-19 13:52:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 10.573526381s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 9.297025299s FAIL (2025-09-19 13:51:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 9.29706237s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.769034782s FAIL (2025-09-19 13:50:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 6.769069178s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 14.539854453s FAIL (2025-09-19 13:50:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 14.53989008s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 17.082770303s FAIL (2025-09-19 13:49:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 17.082800149s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 13.303714661s FAIL (2025-09-19 13:49:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 13.303741041s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 12.06591878s FAIL (2025-09-19 13:49:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 12.065951543s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 10.058566284s FAIL (2025-09-19 13:48:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 10.058610058s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 13.926252222s FAIL (2025-09-19 13:47:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 13.926283412s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 15.462482082s FAIL (2025-09-19 13:46:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 15.462513843s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 14.006658863s FAIL (2025-09-19 13:46:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 14.006695484s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 16.743624202s FAIL (2025-09-19 13:45:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 16.743661705s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 9.941495975s FAIL (2025-09-19 13:44:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 9.941516473s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 18.386951391s FAIL (2025-09-19 13:44:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 18.386986769s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.339063133s FAIL (2025-09-19 13:43:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 6.33909915s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 10.156414386s FAIL (2025-09-19 13:42:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 10.156449222s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 9.112997441s FAIL (2025-09-19 13:42:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 9.113027468s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 16.59526779s FAIL (2025-09-19 13:41:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 16.595298019s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 14.021905167s FAIL (2025-09-19 13:41:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 14.021956134s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 10.941472857s FAIL (2025-09-19 13:40:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 10.94152241s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 18.829091063s FAIL (2025-09-19 13:39:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 18.829134838s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 10.600972898s FAIL (2025-09-19 13:38:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 10.601006803s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 22.466391318s FAIL (2025-09-19 13:38:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 13:33:41.549953136 (312h5m17.589526266s) [upload=274295]: Starting upload Sep 19 13:33:41.550062195 (312h5m17.589635325s) [upload=274295]: Done starting upload Sep 19 13:33:41.550181141 (312h5m17.589754271s) [upload=274295]: Uploading segments... Sep 19 13:33:41.550360302 (312h5m17.589933432s) [upload=274295]: Got next segment. Inline: false Sep 19 13:33:41.550426949 (312h5m17.590000069s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 13:33:41.550433962 (312h5m17.590007092s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 13:33:41.823923431 (312h5m17.863496572s) [seg_pos={0 0}, upload=274295]: Begin upload segment... Sep 19 13:33:41.823987133 (312h5m17.863560263s) [seg_pos={0 0}, upload=274295]: Done begin upload segment. Sep 19 13:33:41.824015177 (312h5m17.863588307s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824024955 (312h5m17.863598095s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.824702475 (312h5m17.864275606s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825095995 (312h5m17.864669115s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.825683193 (312h5m17.865256323s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826101609 (312h5m17.86567473s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.826431157 (312h5m17.866004287s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827055616 (312h5m17.866628746s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827533646 (312h5m17.867106776s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.827794423 (312h5m17.867367543s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828236134 (312h5m17.867809264s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.828677785 (312h5m17.868250915s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829030146 (312h5m17.868603276s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829495732 (312h5m17.869068862s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.829852913 (312h5m17.869426043s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:41.830275317 (312h5m17.869848447s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Uploading piece... Sep 19 13:33:43.581620596 (312h5m19.621193726s) [upload=274295]: Next returned nil segment Sep 19 13:33:43.581645664 (312h5m19.621218784s) [upload=274295]: Waiting for error group managing segments... Sep 19 13:33:43.878141323 (312h5m19.917714453s) [noise=true , address=145.79.227.3:4149 , piece_id=FXBFPFBBFWEAMGCV6HQAGGRBVRAR66OEYQPZA3BCSM5UCTBWKEUA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.933223591 (312h5m19.972796721s) [noise=true , address=145.79.227.14:4837 , piece_id=KP452FMOKBUY2YCIU725TV6HRRBBEGDSGWMVMIFN42SX7LMPLA7Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.946670796 (312h5m19.986243926s) [noise=true , address=145.79.227.6:4259 , piece_id=JZJSUDEDJJOFQXAWFERJJBLX2PHJZNZG4K5BZ47C5TAXWHXSMICQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.952099785 (312h5m19.991672905s) [noise=true , address=45.91.116.8:4047 , piece_id=YR66JJBLWLDLCDRFQRX65I5Q2Q37Y5VTHZY3VXTQCHAMEJE5IEJA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:43.973459728 (312h5m20.013032858s) [noise=true , address=45.91.116.9:4263 , piece_id=QXGG6EONYQVULNG5STLQ27QVFOA6VPMRI2HZKMH64UKISRWXCMRQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.020561530 (312h5m20.06013466s) [noise=true , address=45.91.116.14:4190 , piece_id=UN4RS6PBNKGX6OAZFXYUTEFEYIN72OHLFQVTVBRDXWGFLTREAAZQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.028130162 (312h5m20.067703292s) [noise=true , address=77.90.31.19:4842 , piece_id=V6Z32ZUHSREKDTW6Y7MT3AV2AYKZYACL46CXDGG52AI63BMEBCCA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.029778240 (312h5m20.06935137s) [noise=true , address=77.90.31.11:4554 , piece_id=UB2F6HYE6YAN3ZCUAAGLEKBTL2TLMLY4FILWQ6ISG4F2FTTLI2UA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.030633969 (312h5m20.07020709s) [noise=true , address=77.90.31.12:4592 , piece_id=Q67AJ3DAVLV7ONTMF2JTLGSNQMAQ3KMH5UEUTWOIDFIEUEE45ZMQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.034444396 (312h5m20.074017526s) [noise=true , address=45.91.116.13:4155 , piece_id=JFNFZJAUJ5SNHPCWIDSG5LBNA4EIAZGVNCKDNIWWGM3EBH3BGXBQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.065731273 (312h5m20.105304403s) [noise=true , address=45.91.116.4:4938 , piece_id=Z6JKDV2CUVIN2CHKPLPUVSUKC7BXE2FMROMBVSSTB2T6SD6ERUTA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.088812623 (312h5m20.128385753s) [noise=true , address=77.90.31.13:4627 , piece_id=XFFFDX5IXVSJMNRMVBHFBKNMX3CD62EOXQM3HO3QH3NYO4ADYYMA, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340778529 (312h5m20.380351649s) [noise=true , address=145.79.227.2:4114 , piece_id=6GJVGGAEORJQIIHVWI2PR6C5QIPKMCLNGZ77ATSPO64JXYRTZTAQ, seg_pos={0 0}, upload=274295]: Done uploading piece. err: <nil> Sep 19 13:33:44.340798226 (312h5m20.380371356s) [seg_pos={0 0}, upload=274295]: Segment reached optimal threshold of 13 pieces. Sep 19 13:33:44.340930217 (312h5m20.380503337s) [noise=true , address=145.79.227.19:4079 , piece_id=O2SZHXKFIOFRD2FDQALDFEVMWLSMRDTQJQBYRGRSU6I5HSTY6V5Q, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341011662 (312h5m20.380584792s) [noise=true , address=145.79.227.16:4912 , piece_id=2W3QZSV6OO4CXLD64XEUW2VOIDDAMVHMOBW4XMQCDMTSWA4I6J7A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341086625 (312h5m20.380659755s) [noise=true , address=145.79.227.7:4295 , piece_id=PXCX4UXTWAIFOFEP5RXBIQTOVW4ODREZ5IJ6UCM42WVYKGH2UM6A, seg_pos={0 0}, upload=274295]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Sep 19 13:33:44.341097977 (312h5m20.380671107s) [seg_pos={0 0}, upload=274295]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 13:33:44.341163522 (312h5m20.380736652s) [upload=274295]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 13:33:44.341170044 (312h5m20.380743164s) [upload=274295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 13:33:44.783525219 (312h5m20.823098349s) [upload=274295]: Done uploading segments... Sep 19 13:33:44.783551159 (312h5m20.823124279s) [upload=274295]: Upload finished. err: <nil> Duration: 22.466421145s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)