Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................X...X..................X..............] duration: 13.727071564s FAIL (2025-09-08 16:34:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:22:15.107454512 (50h53m51.147027623s) [upload=44720]: Starting upload Sep 8 16:22:15.107545788 (50h53m51.147118928s) [upload=44720]: Done starting upload Sep 8 16:22:15.107633005 (50h53m51.147206125s) [upload=44720]: Uploading segments... Sep 8 16:22:15.107852124 (50h53m51.147425244s) [upload=44720]: Got next segment. Inline: false Sep 8 16:22:15.107884636 (50h53m51.147457756s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:22:15.107889425 (50h53m51.147462535s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:22:15.656121973 (50h53m51.695695093s) [seg_pos={0 0}, upload=44720]: Begin upload segment... Sep 8 16:22:15.656169475 (50h53m51.695742595s) [seg_pos={0 0}, upload=44720]: Done begin upload segment. Sep 8 16:22:15.656192108 (50h53m51.695765228s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656241372 (50h53m51.695814493s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656577535 (50h53m51.696150655s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656709388 (50h53m51.696282518s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656842883 (50h53m51.696416004s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657232008 (50h53m51.696805128s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657331278 (50h53m51.696904398s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657588310 (50h53m51.69716143s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657772733 (50h53m51.697345863s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657954290 (50h53m51.69752741s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658237763 (50h53m51.697810883s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658338486 (50h53m51.697911606s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658490666 (50h53m51.698063786s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658640233 (50h53m51.698213353s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.661916351 (50h53m51.701489471s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.662318711 (50h53m51.701891831s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:17.470650269 (50h53m53.510223389s) [upload=44720]: Next returned nil segment Sep 8 16:22:17.470667631 (50h53m53.510240751s) [upload=44720]: Waiting for error group managing segments... Sep 8 16:22:17.756531996 (50h53m53.796105116s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.780017016 (50h53m53.819590136s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.798692612 (50h53m53.838265732s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.807587844 (50h53m53.847160954s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.817649647 (50h53m53.857222767s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.839914171 (50h53m53.879487291s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.861697315 (50h53m53.901270655s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.872004868 (50h53m53.911577978s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.939642597 (50h53m53.979215707s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.065684383 (50h53m54.105257503s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.074083004 (50h53m54.113656124s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.085711967 (50h53m54.125285087s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089181967 (50h53m54.128755087s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089206824 (50h53m54.128779944s) [seg_pos={0 0}, upload=44720]: Segment reached optimal threshold of 13 pieces. Sep 8 16:22:18.089320151 (50h53m54.128893271s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089376610 (50h53m54.12894973s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089467584 (50h53m54.129040704s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089520996 (50h53m54.129094116s) [seg_pos={0 0}, upload=44720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:22:18.089622591 (50h53m54.129195711s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:22:18.089629243 (50h53m54.129202373s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:22:18.473990726 (50h53m54.513563846s) [upload=44720]: Done uploading segments... Sep 8 16:22:18.474015564 (50h53m54.513588684s) [upload=44720]: Upload finished. err: <nil> Duration: 13.727091543s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [X......................X...X..................X.............] duration: 7.501855088s FAIL (2025-09-08 16:34:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:22:15.107454512 (50h53m51.147027623s) [upload=44720]: Starting upload Sep 8 16:22:15.107545788 (50h53m51.147118928s) [upload=44720]: Done starting upload Sep 8 16:22:15.107633005 (50h53m51.147206125s) [upload=44720]: Uploading segments... Sep 8 16:22:15.107852124 (50h53m51.147425244s) [upload=44720]: Got next segment. Inline: false Sep 8 16:22:15.107884636 (50h53m51.147457756s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:22:15.107889425 (50h53m51.147462535s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:22:15.656121973 (50h53m51.695695093s) [seg_pos={0 0}, upload=44720]: Begin upload segment... Sep 8 16:22:15.656169475 (50h53m51.695742595s) [seg_pos={0 0}, upload=44720]: Done begin upload segment. Sep 8 16:22:15.656192108 (50h53m51.695765228s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656241372 (50h53m51.695814493s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656577535 (50h53m51.696150655s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656709388 (50h53m51.696282518s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656842883 (50h53m51.696416004s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657232008 (50h53m51.696805128s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657331278 (50h53m51.696904398s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657588310 (50h53m51.69716143s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657772733 (50h53m51.697345863s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657954290 (50h53m51.69752741s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658237763 (50h53m51.697810883s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658338486 (50h53m51.697911606s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658490666 (50h53m51.698063786s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658640233 (50h53m51.698213353s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.661916351 (50h53m51.701489471s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.662318711 (50h53m51.701891831s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:17.470650269 (50h53m53.510223389s) [upload=44720]: Next returned nil segment Sep 8 16:22:17.470667631 (50h53m53.510240751s) [upload=44720]: Waiting for error group managing segments... Sep 8 16:22:17.756531996 (50h53m53.796105116s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.780017016 (50h53m53.819590136s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.798692612 (50h53m53.838265732s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.807587844 (50h53m53.847160954s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.817649647 (50h53m53.857222767s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.839914171 (50h53m53.879487291s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.861697315 (50h53m53.901270655s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.872004868 (50h53m53.911577978s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.939642597 (50h53m53.979215707s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.065684383 (50h53m54.105257503s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.074083004 (50h53m54.113656124s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.085711967 (50h53m54.125285087s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089181967 (50h53m54.128755087s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089206824 (50h53m54.128779944s) [seg_pos={0 0}, upload=44720]: Segment reached optimal threshold of 13 pieces. Sep 8 16:22:18.089320151 (50h53m54.128893271s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089376610 (50h53m54.12894973s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089467584 (50h53m54.129040704s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089520996 (50h53m54.129094116s) [seg_pos={0 0}, upload=44720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:22:18.089622591 (50h53m54.129195711s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:22:18.089629243 (50h53m54.129202373s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:22:18.473990726 (50h53m54.513563846s) [upload=44720]: Done uploading segments... Sep 8 16:22:18.474015564 (50h53m54.513588684s) [upload=44720]: Upload finished. err: <nil> Duration: 7.501877602s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.540612475s FAIL (2025-09-08 16:33:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.540637273s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 7.054680065s FAIL (2025-09-08 16:33:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.054699142s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.X......................X...X..................X............] duration: 6.145739866s FAIL (2025-09-08 16:33:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:22:15.107454512 (50h53m51.147027623s) [upload=44720]: Starting upload Sep 8 16:22:15.107545788 (50h53m51.147118928s) [upload=44720]: Done starting upload Sep 8 16:22:15.107633005 (50h53m51.147206125s) [upload=44720]: Uploading segments... Sep 8 16:22:15.107852124 (50h53m51.147425244s) [upload=44720]: Got next segment. Inline: false Sep 8 16:22:15.107884636 (50h53m51.147457756s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:22:15.107889425 (50h53m51.147462535s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:22:15.656121973 (50h53m51.695695093s) [seg_pos={0 0}, upload=44720]: Begin upload segment... Sep 8 16:22:15.656169475 (50h53m51.695742595s) [seg_pos={0 0}, upload=44720]: Done begin upload segment. Sep 8 16:22:15.656192108 (50h53m51.695765228s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656241372 (50h53m51.695814493s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656577535 (50h53m51.696150655s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656709388 (50h53m51.696282518s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656842883 (50h53m51.696416004s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657232008 (50h53m51.696805128s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657331278 (50h53m51.696904398s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657588310 (50h53m51.69716143s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657772733 (50h53m51.697345863s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657954290 (50h53m51.69752741s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658237763 (50h53m51.697810883s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658338486 (50h53m51.697911606s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658490666 (50h53m51.698063786s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658640233 (50h53m51.698213353s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.661916351 (50h53m51.701489471s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.662318711 (50h53m51.701891831s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:17.470650269 (50h53m53.510223389s) [upload=44720]: Next returned nil segment Sep 8 16:22:17.470667631 (50h53m53.510240751s) [upload=44720]: Waiting for error group managing segments... Sep 8 16:22:17.756531996 (50h53m53.796105116s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.780017016 (50h53m53.819590136s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.798692612 (50h53m53.838265732s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.807587844 (50h53m53.847160954s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.817649647 (50h53m53.857222767s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.839914171 (50h53m53.879487291s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.861697315 (50h53m53.901270655s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.872004868 (50h53m53.911577978s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.939642597 (50h53m53.979215707s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.065684383 (50h53m54.105257503s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.074083004 (50h53m54.113656124s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.085711967 (50h53m54.125285087s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089181967 (50h53m54.128755087s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089206824 (50h53m54.128779944s) [seg_pos={0 0}, upload=44720]: Segment reached optimal threshold of 13 pieces. Sep 8 16:22:18.089320151 (50h53m54.128893271s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089376610 (50h53m54.12894973s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089467584 (50h53m54.129040704s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089520996 (50h53m54.129094116s) [seg_pos={0 0}, upload=44720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:22:18.089622591 (50h53m54.129195711s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:22:18.089629243 (50h53m54.129202373s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:22:18.473990726 (50h53m54.513563846s) [upload=44720]: Done uploading segments... Sep 8 16:22:18.474015564 (50h53m54.513588684s) [upload=44720]: Upload finished. err: <nil> Duration: 6.145759224s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.560713388s FAIL (2025-09-08 16:32:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.56074103s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..X......................X...X..................X...........] duration: 6.701911078s FAIL (2025-09-08 16:32:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:22:15.107454512 (50h53m51.147027623s) [upload=44720]: Starting upload Sep 8 16:22:15.107545788 (50h53m51.147118928s) [upload=44720]: Done starting upload Sep 8 16:22:15.107633005 (50h53m51.147206125s) [upload=44720]: Uploading segments... Sep 8 16:22:15.107852124 (50h53m51.147425244s) [upload=44720]: Got next segment. Inline: false Sep 8 16:22:15.107884636 (50h53m51.147457756s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:22:15.107889425 (50h53m51.147462535s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:22:15.656121973 (50h53m51.695695093s) [seg_pos={0 0}, upload=44720]: Begin upload segment... Sep 8 16:22:15.656169475 (50h53m51.695742595s) [seg_pos={0 0}, upload=44720]: Done begin upload segment. Sep 8 16:22:15.656192108 (50h53m51.695765228s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656241372 (50h53m51.695814493s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656577535 (50h53m51.696150655s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656709388 (50h53m51.696282518s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656842883 (50h53m51.696416004s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657232008 (50h53m51.696805128s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657331278 (50h53m51.696904398s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657588310 (50h53m51.69716143s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657772733 (50h53m51.697345863s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657954290 (50h53m51.69752741s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658237763 (50h53m51.697810883s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658338486 (50h53m51.697911606s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658490666 (50h53m51.698063786s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658640233 (50h53m51.698213353s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.661916351 (50h53m51.701489471s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.662318711 (50h53m51.701891831s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:17.470650269 (50h53m53.510223389s) [upload=44720]: Next returned nil segment Sep 8 16:22:17.470667631 (50h53m53.510240751s) [upload=44720]: Waiting for error group managing segments... Sep 8 16:22:17.756531996 (50h53m53.796105116s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.780017016 (50h53m53.819590136s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.798692612 (50h53m53.838265732s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.807587844 (50h53m53.847160954s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.817649647 (50h53m53.857222767s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.839914171 (50h53m53.879487291s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.861697315 (50h53m53.901270655s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.872004868 (50h53m53.911577978s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.939642597 (50h53m53.979215707s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.065684383 (50h53m54.105257503s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.074083004 (50h53m54.113656124s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.085711967 (50h53m54.125285087s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089181967 (50h53m54.128755087s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089206824 (50h53m54.128779944s) [seg_pos={0 0}, upload=44720]: Segment reached optimal threshold of 13 pieces. Sep 8 16:22:18.089320151 (50h53m54.128893271s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089376610 (50h53m54.12894973s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089467584 (50h53m54.129040704s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089520996 (50h53m54.129094116s) [seg_pos={0 0}, upload=44720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:22:18.089622591 (50h53m54.129195711s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:22:18.089629243 (50h53m54.129202373s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:22:18.473990726 (50h53m54.513563846s) [upload=44720]: Done uploading segments... Sep 8 16:22:18.474015564 (50h53m54.513588684s) [upload=44720]: Upload finished. err: <nil> Duration: 6.70193882s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.714404055s FAIL (2025-09-08 16:32:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.714423702s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...X......................X...X..................X..........] duration: 15.08422332s FAIL (2025-09-08 16:31:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:22:15.107454512 (50h53m51.147027623s) [upload=44720]: Starting upload Sep 8 16:22:15.107545788 (50h53m51.147118928s) [upload=44720]: Done starting upload Sep 8 16:22:15.107633005 (50h53m51.147206125s) [upload=44720]: Uploading segments... Sep 8 16:22:15.107852124 (50h53m51.147425244s) [upload=44720]: Got next segment. Inline: false Sep 8 16:22:15.107884636 (50h53m51.147457756s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:22:15.107889425 (50h53m51.147462535s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:22:15.656121973 (50h53m51.695695093s) [seg_pos={0 0}, upload=44720]: Begin upload segment... Sep 8 16:22:15.656169475 (50h53m51.695742595s) [seg_pos={0 0}, upload=44720]: Done begin upload segment. Sep 8 16:22:15.656192108 (50h53m51.695765228s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656241372 (50h53m51.695814493s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656577535 (50h53m51.696150655s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656709388 (50h53m51.696282518s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656842883 (50h53m51.696416004s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657232008 (50h53m51.696805128s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657331278 (50h53m51.696904398s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657588310 (50h53m51.69716143s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657772733 (50h53m51.697345863s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657954290 (50h53m51.69752741s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658237763 (50h53m51.697810883s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658338486 (50h53m51.697911606s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658490666 (50h53m51.698063786s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658640233 (50h53m51.698213353s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.661916351 (50h53m51.701489471s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.662318711 (50h53m51.701891831s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:17.470650269 (50h53m53.510223389s) [upload=44720]: Next returned nil segment Sep 8 16:22:17.470667631 (50h53m53.510240751s) [upload=44720]: Waiting for error group managing segments... Sep 8 16:22:17.756531996 (50h53m53.796105116s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.780017016 (50h53m53.819590136s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.798692612 (50h53m53.838265732s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.807587844 (50h53m53.847160954s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.817649647 (50h53m53.857222767s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.839914171 (50h53m53.879487291s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.861697315 (50h53m53.901270655s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.872004868 (50h53m53.911577978s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.939642597 (50h53m53.979215707s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.065684383 (50h53m54.105257503s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.074083004 (50h53m54.113656124s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.085711967 (50h53m54.125285087s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089181967 (50h53m54.128755087s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089206824 (50h53m54.128779944s) [seg_pos={0 0}, upload=44720]: Segment reached optimal threshold of 13 pieces. Sep 8 16:22:18.089320151 (50h53m54.128893271s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089376610 (50h53m54.12894973s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089467584 (50h53m54.129040704s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089520996 (50h53m54.129094116s) [seg_pos={0 0}, upload=44720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:22:18.089622591 (50h53m54.129195711s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:22:18.089629243 (50h53m54.129202373s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:22:18.473990726 (50h53m54.513563846s) [upload=44720]: Done uploading segments... Sep 8 16:22:18.474015564 (50h53m54.513588684s) [upload=44720]: Upload finished. err: <nil> Duration: 15.084255713s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X..........X................................................] duration: 6.566822006s FAIL (2025-09-08 16:31:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.566845711s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [XX..........X...............................................] duration: 6.58022605s FAIL (2025-09-08 16:30:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.580250206s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....X......................X...X..................X.........] duration: 6.08948918s FAIL (2025-09-08 16:30:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:22:15.107454512 (50h53m51.147027623s) [upload=44720]: Starting upload Sep 8 16:22:15.107545788 (50h53m51.147118928s) [upload=44720]: Done starting upload Sep 8 16:22:15.107633005 (50h53m51.147206125s) [upload=44720]: Uploading segments... Sep 8 16:22:15.107852124 (50h53m51.147425244s) [upload=44720]: Got next segment. Inline: false Sep 8 16:22:15.107884636 (50h53m51.147457756s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:22:15.107889425 (50h53m51.147462535s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:22:15.656121973 (50h53m51.695695093s) [seg_pos={0 0}, upload=44720]: Begin upload segment... Sep 8 16:22:15.656169475 (50h53m51.695742595s) [seg_pos={0 0}, upload=44720]: Done begin upload segment. Sep 8 16:22:15.656192108 (50h53m51.695765228s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656241372 (50h53m51.695814493s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656577535 (50h53m51.696150655s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656709388 (50h53m51.696282518s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656842883 (50h53m51.696416004s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657232008 (50h53m51.696805128s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657331278 (50h53m51.696904398s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657588310 (50h53m51.69716143s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657772733 (50h53m51.697345863s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657954290 (50h53m51.69752741s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658237763 (50h53m51.697810883s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658338486 (50h53m51.697911606s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658490666 (50h53m51.698063786s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658640233 (50h53m51.698213353s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.661916351 (50h53m51.701489471s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.662318711 (50h53m51.701891831s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:17.470650269 (50h53m53.510223389s) [upload=44720]: Next returned nil segment Sep 8 16:22:17.470667631 (50h53m53.510240751s) [upload=44720]: Waiting for error group managing segments... Sep 8 16:22:17.756531996 (50h53m53.796105116s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.780017016 (50h53m53.819590136s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.798692612 (50h53m53.838265732s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.807587844 (50h53m53.847160954s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.817649647 (50h53m53.857222767s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.839914171 (50h53m53.879487291s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.861697315 (50h53m53.901270655s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.872004868 (50h53m53.911577978s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.939642597 (50h53m53.979215707s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.065684383 (50h53m54.105257503s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.074083004 (50h53m54.113656124s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.085711967 (50h53m54.125285087s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089181967 (50h53m54.128755087s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089206824 (50h53m54.128779944s) [seg_pos={0 0}, upload=44720]: Segment reached optimal threshold of 13 pieces. Sep 8 16:22:18.089320151 (50h53m54.128893271s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089376610 (50h53m54.12894973s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089467584 (50h53m54.129040704s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089520996 (50h53m54.129094116s) [seg_pos={0 0}, upload=44720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:22:18.089622591 (50h53m54.129195711s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:22:18.089629243 (50h53m54.129202373s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:22:18.473990726 (50h53m54.513563846s) [upload=44720]: Done uploading segments... Sep 8 16:22:18.474015564 (50h53m54.513588684s) [upload=44720]: Upload finished. err: <nil> Duration: 6.089515379s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [XXX..........X..............................................] duration: 6.917224325s FAIL (2025-09-08 16:30:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.917265245s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....X......................X...X..................X........] duration: 7.395002411s FAIL (2025-09-08 16:29:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:22:15.107454512 (50h53m51.147027623s) [upload=44720]: Starting upload Sep 8 16:22:15.107545788 (50h53m51.147118928s) [upload=44720]: Done starting upload Sep 8 16:22:15.107633005 (50h53m51.147206125s) [upload=44720]: Uploading segments... Sep 8 16:22:15.107852124 (50h53m51.147425244s) [upload=44720]: Got next segment. Inline: false Sep 8 16:22:15.107884636 (50h53m51.147457756s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:22:15.107889425 (50h53m51.147462535s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:22:15.656121973 (50h53m51.695695093s) [seg_pos={0 0}, upload=44720]: Begin upload segment... Sep 8 16:22:15.656169475 (50h53m51.695742595s) [seg_pos={0 0}, upload=44720]: Done begin upload segment. Sep 8 16:22:15.656192108 (50h53m51.695765228s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656241372 (50h53m51.695814493s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656577535 (50h53m51.696150655s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656709388 (50h53m51.696282518s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656842883 (50h53m51.696416004s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657232008 (50h53m51.696805128s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657331278 (50h53m51.696904398s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657588310 (50h53m51.69716143s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657772733 (50h53m51.697345863s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657954290 (50h53m51.69752741s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658237763 (50h53m51.697810883s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658338486 (50h53m51.697911606s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658490666 (50h53m51.698063786s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658640233 (50h53m51.698213353s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.661916351 (50h53m51.701489471s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.662318711 (50h53m51.701891831s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:17.470650269 (50h53m53.510223389s) [upload=44720]: Next returned nil segment Sep 8 16:22:17.470667631 (50h53m53.510240751s) [upload=44720]: Waiting for error group managing segments... Sep 8 16:22:17.756531996 (50h53m53.796105116s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.780017016 (50h53m53.819590136s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.798692612 (50h53m53.838265732s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.807587844 (50h53m53.847160954s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.817649647 (50h53m53.857222767s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.839914171 (50h53m53.879487291s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.861697315 (50h53m53.901270655s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.872004868 (50h53m53.911577978s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.939642597 (50h53m53.979215707s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.065684383 (50h53m54.105257503s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.074083004 (50h53m54.113656124s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.085711967 (50h53m54.125285087s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089181967 (50h53m54.128755087s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089206824 (50h53m54.128779944s) [seg_pos={0 0}, upload=44720]: Segment reached optimal threshold of 13 pieces. Sep 8 16:22:18.089320151 (50h53m54.128893271s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089376610 (50h53m54.12894973s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089467584 (50h53m54.129040704s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089520996 (50h53m54.129094116s) [seg_pos={0 0}, upload=44720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:22:18.089622591 (50h53m54.129195711s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:22:18.089629243 (50h53m54.129202373s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:22:18.473990726 (50h53m54.513563846s) [upload=44720]: Done uploading segments... Sep 8 16:22:18.474015564 (50h53m54.513588684s) [upload=44720]: Upload finished. err: <nil> Duration: 7.395027268s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.XXX..........X.............................................] duration: 6.622581466s FAIL (2025-09-08 16:29:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.622606765s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..XXX..........X............................................] duration: 6.798176508s FAIL (2025-09-08 16:28:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.798199992s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......X......................X...X..................X.......] duration: 6.469890118s FAIL (2025-09-08 16:28:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:22:15.107454512 (50h53m51.147027623s) [upload=44720]: Starting upload Sep 8 16:22:15.107545788 (50h53m51.147118928s) [upload=44720]: Done starting upload Sep 8 16:22:15.107633005 (50h53m51.147206125s) [upload=44720]: Uploading segments... Sep 8 16:22:15.107852124 (50h53m51.147425244s) [upload=44720]: Got next segment. Inline: false Sep 8 16:22:15.107884636 (50h53m51.147457756s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:22:15.107889425 (50h53m51.147462535s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:22:15.656121973 (50h53m51.695695093s) [seg_pos={0 0}, upload=44720]: Begin upload segment... Sep 8 16:22:15.656169475 (50h53m51.695742595s) [seg_pos={0 0}, upload=44720]: Done begin upload segment. Sep 8 16:22:15.656192108 (50h53m51.695765228s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656241372 (50h53m51.695814493s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656577535 (50h53m51.696150655s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656709388 (50h53m51.696282518s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656842883 (50h53m51.696416004s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657232008 (50h53m51.696805128s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657331278 (50h53m51.696904398s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657588310 (50h53m51.69716143s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657772733 (50h53m51.697345863s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657954290 (50h53m51.69752741s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658237763 (50h53m51.697810883s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658338486 (50h53m51.697911606s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658490666 (50h53m51.698063786s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658640233 (50h53m51.698213353s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.661916351 (50h53m51.701489471s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.662318711 (50h53m51.701891831s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:17.470650269 (50h53m53.510223389s) [upload=44720]: Next returned nil segment Sep 8 16:22:17.470667631 (50h53m53.510240751s) [upload=44720]: Waiting for error group managing segments... Sep 8 16:22:17.756531996 (50h53m53.796105116s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.780017016 (50h53m53.819590136s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.798692612 (50h53m53.838265732s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.807587844 (50h53m53.847160954s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.817649647 (50h53m53.857222767s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.839914171 (50h53m53.879487291s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.861697315 (50h53m53.901270655s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.872004868 (50h53m53.911577978s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.939642597 (50h53m53.979215707s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.065684383 (50h53m54.105257503s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.074083004 (50h53m54.113656124s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.085711967 (50h53m54.125285087s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089181967 (50h53m54.128755087s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089206824 (50h53m54.128779944s) [seg_pos={0 0}, upload=44720]: Segment reached optimal threshold of 13 pieces. Sep 8 16:22:18.089320151 (50h53m54.128893271s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089376610 (50h53m54.12894973s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089467584 (50h53m54.129040704s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089520996 (50h53m54.129094116s) [seg_pos={0 0}, upload=44720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:22:18.089622591 (50h53m54.129195711s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:22:18.089629243 (50h53m54.129202373s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:22:18.473990726 (50h53m54.513563846s) [upload=44720]: Done uploading segments... Sep 8 16:22:18.474015564 (50h53m54.513588684s) [upload=44720]: Upload finished. err: <nil> Duration: 6.469919943s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [X..XXX..........X...........................................] duration: 6.210397782s FAIL (2025-09-08 16:28:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.210420826s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......X......................X...X..................X......] duration: 6.206563816s FAIL (2025-09-08 16:27:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:22:15.107454512 (50h53m51.147027623s) [upload=44720]: Starting upload Sep 8 16:22:15.107545788 (50h53m51.147118928s) [upload=44720]: Done starting upload Sep 8 16:22:15.107633005 (50h53m51.147206125s) [upload=44720]: Uploading segments... Sep 8 16:22:15.107852124 (50h53m51.147425244s) [upload=44720]: Got next segment. Inline: false Sep 8 16:22:15.107884636 (50h53m51.147457756s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:22:15.107889425 (50h53m51.147462535s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:22:15.656121973 (50h53m51.695695093s) [seg_pos={0 0}, upload=44720]: Begin upload segment... Sep 8 16:22:15.656169475 (50h53m51.695742595s) [seg_pos={0 0}, upload=44720]: Done begin upload segment. Sep 8 16:22:15.656192108 (50h53m51.695765228s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656241372 (50h53m51.695814493s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656577535 (50h53m51.696150655s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656709388 (50h53m51.696282518s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656842883 (50h53m51.696416004s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657232008 (50h53m51.696805128s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657331278 (50h53m51.696904398s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657588310 (50h53m51.69716143s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657772733 (50h53m51.697345863s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657954290 (50h53m51.69752741s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658237763 (50h53m51.697810883s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658338486 (50h53m51.697911606s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658490666 (50h53m51.698063786s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658640233 (50h53m51.698213353s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.661916351 (50h53m51.701489471s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.662318711 (50h53m51.701891831s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:17.470650269 (50h53m53.510223389s) [upload=44720]: Next returned nil segment Sep 8 16:22:17.470667631 (50h53m53.510240751s) [upload=44720]: Waiting for error group managing segments... Sep 8 16:22:17.756531996 (50h53m53.796105116s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.780017016 (50h53m53.819590136s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.798692612 (50h53m53.838265732s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.807587844 (50h53m53.847160954s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.817649647 (50h53m53.857222767s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.839914171 (50h53m53.879487291s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.861697315 (50h53m53.901270655s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.872004868 (50h53m53.911577978s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.939642597 (50h53m53.979215707s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.065684383 (50h53m54.105257503s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.074083004 (50h53m54.113656124s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.085711967 (50h53m54.125285087s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089181967 (50h53m54.128755087s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089206824 (50h53m54.128779944s) [seg_pos={0 0}, upload=44720]: Segment reached optimal threshold of 13 pieces. Sep 8 16:22:18.089320151 (50h53m54.128893271s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089376610 (50h53m54.12894973s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089467584 (50h53m54.129040704s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089520996 (50h53m54.129094116s) [seg_pos={0 0}, upload=44720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:22:18.089622591 (50h53m54.129195711s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:22:18.089629243 (50h53m54.129202373s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:22:18.473990726 (50h53m54.513563846s) [upload=44720]: Done uploading segments... Sep 8 16:22:18.474015564 (50h53m54.513588684s) [upload=44720]: Upload finished. err: <nil> Duration: 6.206600336s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.X..XXX..........X..........................................] duration: 6.440618612s FAIL (2025-09-08 16:27:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.440640894s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........X......................X...X..................X.....] duration: 7.769874186s FAIL (2025-09-08 16:26:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:22:15.107454512 (50h53m51.147027623s) [upload=44720]: Starting upload Sep 8 16:22:15.107545788 (50h53m51.147118928s) [upload=44720]: Done starting upload Sep 8 16:22:15.107633005 (50h53m51.147206125s) [upload=44720]: Uploading segments... Sep 8 16:22:15.107852124 (50h53m51.147425244s) [upload=44720]: Got next segment. Inline: false Sep 8 16:22:15.107884636 (50h53m51.147457756s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:22:15.107889425 (50h53m51.147462535s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:22:15.656121973 (50h53m51.695695093s) [seg_pos={0 0}, upload=44720]: Begin upload segment... Sep 8 16:22:15.656169475 (50h53m51.695742595s) [seg_pos={0 0}, upload=44720]: Done begin upload segment. Sep 8 16:22:15.656192108 (50h53m51.695765228s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656241372 (50h53m51.695814493s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656577535 (50h53m51.696150655s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656709388 (50h53m51.696282518s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656842883 (50h53m51.696416004s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657232008 (50h53m51.696805128s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657331278 (50h53m51.696904398s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657588310 (50h53m51.69716143s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657772733 (50h53m51.697345863s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657954290 (50h53m51.69752741s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658237763 (50h53m51.697810883s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658338486 (50h53m51.697911606s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658490666 (50h53m51.698063786s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658640233 (50h53m51.698213353s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.661916351 (50h53m51.701489471s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.662318711 (50h53m51.701891831s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:17.470650269 (50h53m53.510223389s) [upload=44720]: Next returned nil segment Sep 8 16:22:17.470667631 (50h53m53.510240751s) [upload=44720]: Waiting for error group managing segments... Sep 8 16:22:17.756531996 (50h53m53.796105116s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.780017016 (50h53m53.819590136s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.798692612 (50h53m53.838265732s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.807587844 (50h53m53.847160954s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.817649647 (50h53m53.857222767s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.839914171 (50h53m53.879487291s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.861697315 (50h53m53.901270655s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.872004868 (50h53m53.911577978s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.939642597 (50h53m53.979215707s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.065684383 (50h53m54.105257503s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.074083004 (50h53m54.113656124s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.085711967 (50h53m54.125285087s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089181967 (50h53m54.128755087s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089206824 (50h53m54.128779944s) [seg_pos={0 0}, upload=44720]: Segment reached optimal threshold of 13 pieces. Sep 8 16:22:18.089320151 (50h53m54.128893271s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089376610 (50h53m54.12894973s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089467584 (50h53m54.129040704s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089520996 (50h53m54.129094116s) [seg_pos={0 0}, upload=44720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:22:18.089622591 (50h53m54.129195711s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:22:18.089629243 (50h53m54.129202373s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:22:18.473990726 (50h53m54.513563846s) [upload=44720]: Done uploading segments... Sep 8 16:22:18.474015564 (50h53m54.513588684s) [upload=44720]: Upload finished. err: <nil> Duration: 7.769902109s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..X..XXX..........X.........................................] duration: 6.04679953s FAIL (2025-09-08 16:26:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.046824356s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........X......................X...X..................X....] duration: 6.541935165s FAIL (2025-09-08 16:25:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:22:15.107454512 (50h53m51.147027623s) [upload=44720]: Starting upload Sep 8 16:22:15.107545788 (50h53m51.147118928s) [upload=44720]: Done starting upload Sep 8 16:22:15.107633005 (50h53m51.147206125s) [upload=44720]: Uploading segments... Sep 8 16:22:15.107852124 (50h53m51.147425244s) [upload=44720]: Got next segment. Inline: false Sep 8 16:22:15.107884636 (50h53m51.147457756s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:22:15.107889425 (50h53m51.147462535s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:22:15.656121973 (50h53m51.695695093s) [seg_pos={0 0}, upload=44720]: Begin upload segment... Sep 8 16:22:15.656169475 (50h53m51.695742595s) [seg_pos={0 0}, upload=44720]: Done begin upload segment. Sep 8 16:22:15.656192108 (50h53m51.695765228s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656241372 (50h53m51.695814493s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656577535 (50h53m51.696150655s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656709388 (50h53m51.696282518s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656842883 (50h53m51.696416004s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657232008 (50h53m51.696805128s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657331278 (50h53m51.696904398s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657588310 (50h53m51.69716143s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657772733 (50h53m51.697345863s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657954290 (50h53m51.69752741s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658237763 (50h53m51.697810883s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658338486 (50h53m51.697911606s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658490666 (50h53m51.698063786s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658640233 (50h53m51.698213353s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.661916351 (50h53m51.701489471s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.662318711 (50h53m51.701891831s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:17.470650269 (50h53m53.510223389s) [upload=44720]: Next returned nil segment Sep 8 16:22:17.470667631 (50h53m53.510240751s) [upload=44720]: Waiting for error group managing segments... Sep 8 16:22:17.756531996 (50h53m53.796105116s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.780017016 (50h53m53.819590136s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.798692612 (50h53m53.838265732s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.807587844 (50h53m53.847160954s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.817649647 (50h53m53.857222767s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.839914171 (50h53m53.879487291s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.861697315 (50h53m53.901270655s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.872004868 (50h53m53.911577978s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.939642597 (50h53m53.979215707s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.065684383 (50h53m54.105257503s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.074083004 (50h53m54.113656124s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.085711967 (50h53m54.125285087s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089181967 (50h53m54.128755087s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089206824 (50h53m54.128779944s) [seg_pos={0 0}, upload=44720]: Segment reached optimal threshold of 13 pieces. Sep 8 16:22:18.089320151 (50h53m54.128893271s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089376610 (50h53m54.12894973s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089467584 (50h53m54.129040704s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089520996 (50h53m54.129094116s) [seg_pos={0 0}, upload=44720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:22:18.089622591 (50h53m54.129195711s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:22:18.089629243 (50h53m54.129202373s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:22:18.473990726 (50h53m54.513563846s) [upload=44720]: Done uploading segments... Sep 8 16:22:18.474015564 (50h53m54.513588684s) [upload=44720]: Upload finished. err: <nil> Duration: 6.541960233s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...X..XXX..........X........................................] duration: 6.418190055s FAIL (2025-09-08 16:25:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.418286419s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........X......................X...X..................X...] duration: 6.099715648s FAIL (2025-09-08 16:25:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:22:15.107454512 (50h53m51.147027623s) [upload=44720]: Starting upload Sep 8 16:22:15.107545788 (50h53m51.147118928s) [upload=44720]: Done starting upload Sep 8 16:22:15.107633005 (50h53m51.147206125s) [upload=44720]: Uploading segments... Sep 8 16:22:15.107852124 (50h53m51.147425244s) [upload=44720]: Got next segment. Inline: false Sep 8 16:22:15.107884636 (50h53m51.147457756s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:22:15.107889425 (50h53m51.147462535s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:22:15.656121973 (50h53m51.695695093s) [seg_pos={0 0}, upload=44720]: Begin upload segment... Sep 8 16:22:15.656169475 (50h53m51.695742595s) [seg_pos={0 0}, upload=44720]: Done begin upload segment. Sep 8 16:22:15.656192108 (50h53m51.695765228s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656241372 (50h53m51.695814493s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656577535 (50h53m51.696150655s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656709388 (50h53m51.696282518s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656842883 (50h53m51.696416004s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657232008 (50h53m51.696805128s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657331278 (50h53m51.696904398s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657588310 (50h53m51.69716143s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657772733 (50h53m51.697345863s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657954290 (50h53m51.69752741s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658237763 (50h53m51.697810883s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658338486 (50h53m51.697911606s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658490666 (50h53m51.698063786s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658640233 (50h53m51.698213353s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.661916351 (50h53m51.701489471s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.662318711 (50h53m51.701891831s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:17.470650269 (50h53m53.510223389s) [upload=44720]: Next returned nil segment Sep 8 16:22:17.470667631 (50h53m53.510240751s) [upload=44720]: Waiting for error group managing segments... Sep 8 16:22:17.756531996 (50h53m53.796105116s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.780017016 (50h53m53.819590136s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.798692612 (50h53m53.838265732s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.807587844 (50h53m53.847160954s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.817649647 (50h53m53.857222767s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.839914171 (50h53m53.879487291s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.861697315 (50h53m53.901270655s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.872004868 (50h53m53.911577978s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.939642597 (50h53m53.979215707s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.065684383 (50h53m54.105257503s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.074083004 (50h53m54.113656124s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.085711967 (50h53m54.125285087s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089181967 (50h53m54.128755087s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089206824 (50h53m54.128779944s) [seg_pos={0 0}, upload=44720]: Segment reached optimal threshold of 13 pieces. Sep 8 16:22:18.089320151 (50h53m54.128893271s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089376610 (50h53m54.12894973s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089467584 (50h53m54.129040704s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089520996 (50h53m54.129094116s) [seg_pos={0 0}, upload=44720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:22:18.089622591 (50h53m54.129195711s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:22:18.089629243 (50h53m54.129202373s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:22:18.473990726 (50h53m54.513563846s) [upload=44720]: Done uploading segments... Sep 8 16:22:18.474015564 (50h53m54.513588684s) [upload=44720]: Upload finished. err: <nil> Duration: 6.099744594s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [....X..XXX..........X.......................................] duration: 6.605122558s FAIL (2025-09-08 16:24:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.605467037s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........X......................X...X..................X..] duration: 7.041955997s FAIL (2025-09-08 16:24:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:22:15.107454512 (50h53m51.147027623s) [upload=44720]: Starting upload Sep 8 16:22:15.107545788 (50h53m51.147118928s) [upload=44720]: Done starting upload Sep 8 16:22:15.107633005 (50h53m51.147206125s) [upload=44720]: Uploading segments... Sep 8 16:22:15.107852124 (50h53m51.147425244s) [upload=44720]: Got next segment. Inline: false Sep 8 16:22:15.107884636 (50h53m51.147457756s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:22:15.107889425 (50h53m51.147462535s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:22:15.656121973 (50h53m51.695695093s) [seg_pos={0 0}, upload=44720]: Begin upload segment... Sep 8 16:22:15.656169475 (50h53m51.695742595s) [seg_pos={0 0}, upload=44720]: Done begin upload segment. Sep 8 16:22:15.656192108 (50h53m51.695765228s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656241372 (50h53m51.695814493s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656577535 (50h53m51.696150655s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656709388 (50h53m51.696282518s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656842883 (50h53m51.696416004s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657232008 (50h53m51.696805128s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657331278 (50h53m51.696904398s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657588310 (50h53m51.69716143s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657772733 (50h53m51.697345863s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657954290 (50h53m51.69752741s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658237763 (50h53m51.697810883s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658338486 (50h53m51.697911606s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658490666 (50h53m51.698063786s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658640233 (50h53m51.698213353s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.661916351 (50h53m51.701489471s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.662318711 (50h53m51.701891831s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:17.470650269 (50h53m53.510223389s) [upload=44720]: Next returned nil segment Sep 8 16:22:17.470667631 (50h53m53.510240751s) [upload=44720]: Waiting for error group managing segments... Sep 8 16:22:17.756531996 (50h53m53.796105116s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.780017016 (50h53m53.819590136s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.798692612 (50h53m53.838265732s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.807587844 (50h53m53.847160954s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.817649647 (50h53m53.857222767s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.839914171 (50h53m53.879487291s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.861697315 (50h53m53.901270655s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.872004868 (50h53m53.911577978s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.939642597 (50h53m53.979215707s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.065684383 (50h53m54.105257503s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.074083004 (50h53m54.113656124s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.085711967 (50h53m54.125285087s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089181967 (50h53m54.128755087s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089206824 (50h53m54.128779944s) [seg_pos={0 0}, upload=44720]: Segment reached optimal threshold of 13 pieces. Sep 8 16:22:18.089320151 (50h53m54.128893271s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089376610 (50h53m54.12894973s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089467584 (50h53m54.129040704s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089520996 (50h53m54.129094116s) [seg_pos={0 0}, upload=44720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:22:18.089622591 (50h53m54.129195711s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:22:18.089629243 (50h53m54.129202373s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:22:18.473990726 (50h53m54.513563846s) [upload=44720]: Done uploading segments... Sep 8 16:22:18.474015564 (50h53m54.513588684s) [upload=44720]: Upload finished. err: <nil> Duration: 7.041980885s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.....X..XXX..........X......................................] duration: 6.62013837s FAIL (2025-09-08 16:23:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.62016397s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [............X......................X...X..................X.] duration: 6.202940183s FAIL (2025-09-08 16:23:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:22:15.107454512 (50h53m51.147027623s) [upload=44720]: Starting upload Sep 8 16:22:15.107545788 (50h53m51.147118928s) [upload=44720]: Done starting upload Sep 8 16:22:15.107633005 (50h53m51.147206125s) [upload=44720]: Uploading segments... Sep 8 16:22:15.107852124 (50h53m51.147425244s) [upload=44720]: Got next segment. Inline: false Sep 8 16:22:15.107884636 (50h53m51.147457756s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:22:15.107889425 (50h53m51.147462535s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:22:15.656121973 (50h53m51.695695093s) [seg_pos={0 0}, upload=44720]: Begin upload segment... Sep 8 16:22:15.656169475 (50h53m51.695742595s) [seg_pos={0 0}, upload=44720]: Done begin upload segment. Sep 8 16:22:15.656192108 (50h53m51.695765228s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656241372 (50h53m51.695814493s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656577535 (50h53m51.696150655s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656709388 (50h53m51.696282518s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656842883 (50h53m51.696416004s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657232008 (50h53m51.696805128s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657331278 (50h53m51.696904398s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657588310 (50h53m51.69716143s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657772733 (50h53m51.697345863s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657954290 (50h53m51.69752741s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658237763 (50h53m51.697810883s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658338486 (50h53m51.697911606s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658490666 (50h53m51.698063786s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658640233 (50h53m51.698213353s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.661916351 (50h53m51.701489471s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.662318711 (50h53m51.701891831s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:17.470650269 (50h53m53.510223389s) [upload=44720]: Next returned nil segment Sep 8 16:22:17.470667631 (50h53m53.510240751s) [upload=44720]: Waiting for error group managing segments... Sep 8 16:22:17.756531996 (50h53m53.796105116s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.780017016 (50h53m53.819590136s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.798692612 (50h53m53.838265732s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.807587844 (50h53m53.847160954s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.817649647 (50h53m53.857222767s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.839914171 (50h53m53.879487291s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.861697315 (50h53m53.901270655s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.872004868 (50h53m53.911577978s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.939642597 (50h53m53.979215707s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.065684383 (50h53m54.105257503s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.074083004 (50h53m54.113656124s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.085711967 (50h53m54.125285087s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089181967 (50h53m54.128755087s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089206824 (50h53m54.128779944s) [seg_pos={0 0}, upload=44720]: Segment reached optimal threshold of 13 pieces. Sep 8 16:22:18.089320151 (50h53m54.128893271s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089376610 (50h53m54.12894973s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089467584 (50h53m54.129040704s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089520996 (50h53m54.129094116s) [seg_pos={0 0}, upload=44720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:22:18.089622591 (50h53m54.129195711s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:22:18.089629243 (50h53m54.129202373s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:22:18.473990726 (50h53m54.513563846s) [upload=44720]: Done uploading segments... Sep 8 16:22:18.474015564 (50h53m54.513588684s) [upload=44720]: Upload finished. err: <nil> Duration: 6.202967745s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [......X..XXX..........X.....................................] duration: 6.865982088s FAIL (2025-09-08 16:23:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.866004983s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............X......................X...X..................X] duration: 30.002164911s FAIL (2025-09-08 16:22:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:22:15.107454512 (50h53m51.147027623s) [upload=44720]: Starting upload Sep 8 16:22:15.107545788 (50h53m51.147118928s) [upload=44720]: Done starting upload Sep 8 16:22:15.107633005 (50h53m51.147206125s) [upload=44720]: Uploading segments... Sep 8 16:22:15.107852124 (50h53m51.147425244s) [upload=44720]: Got next segment. Inline: false Sep 8 16:22:15.107884636 (50h53m51.147457756s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:22:15.107889425 (50h53m51.147462535s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:22:15.656121973 (50h53m51.695695093s) [seg_pos={0 0}, upload=44720]: Begin upload segment... Sep 8 16:22:15.656169475 (50h53m51.695742595s) [seg_pos={0 0}, upload=44720]: Done begin upload segment. Sep 8 16:22:15.656192108 (50h53m51.695765228s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656241372 (50h53m51.695814493s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656577535 (50h53m51.696150655s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656709388 (50h53m51.696282518s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.656842883 (50h53m51.696416004s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657232008 (50h53m51.696805128s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657331278 (50h53m51.696904398s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657588310 (50h53m51.69716143s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657772733 (50h53m51.697345863s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.657954290 (50h53m51.69752741s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658237763 (50h53m51.697810883s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658338486 (50h53m51.697911606s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658490666 (50h53m51.698063786s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.658640233 (50h53m51.698213353s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.661916351 (50h53m51.701489471s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:15.662318711 (50h53m51.701891831s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Uploading piece... Sep 8 16:22:17.470650269 (50h53m53.510223389s) [upload=44720]: Next returned nil segment Sep 8 16:22:17.470667631 (50h53m53.510240751s) [upload=44720]: Waiting for error group managing segments... Sep 8 16:22:17.756531996 (50h53m53.796105116s) [noise=true , address=45.91.116.12:4226 , piece_id=JJJFFUVHFT2TJTONYVIVYL7PED47NA77DUEBIYTIRWZAPED5KOTA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.780017016 (50h53m53.819590136s) [noise=true , address=45.91.116.14:4191 , piece_id=4MRNLWWW3K4Z6I4L57E7S7RCCMOEJKLHICYJZULW3MMXS6A65FNQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.798692612 (50h53m53.838265732s) [noise=true , address=45.91.116.5:4297 , piece_id=3GIYXH77AJEQJJ5XLLDTKGKGB64GLRRFC3YDJNJZ4YL5SOOB5MFQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.807587844 (50h53m53.847160954s) [noise=true , address=45.91.116.10:4081 , piece_id=KMGPWNVEV5LDWDBEIIEZ6T6MZFPCVN3YGV6UOVDDDTW3DUUUNLQA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.817649647 (50h53m53.857222767s) [noise=true , address=77.90.31.14:4662 , piece_id=UHECF5OL6MWZQ6D2EZCYLHQ7JOZOXRMI25VXGUYPLIVB4RKFO6VA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.839914171 (50h53m53.879487291s) [noise=true , address=45.91.116.4:4940 , piece_id=JG2PTSVTNPEEVEZSQP47GWW4UWVBNV2XNT3GLB7BPXWBMYMMRNBA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.861697315 (50h53m53.901270655s) [noise=true , address=145.79.227.19:4077 , piece_id=O25DO73JQOU57ZV5VVPWYYQEBKVKYZESGRPGQNR6PR77DGX5GGSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.872004868 (50h53m53.911577978s) [noise=true , address=77.90.31.12:4590 , piece_id=NRR3TK45H5DPKI75YTV667DO76U2PF6DCVEAX3J6E6PHQSZKKNAQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:17.939642597 (50h53m53.979215707s) [noise=true , address=145.79.227.3:4149 , piece_id=GYLPVZS7SYRWHY5XAPKXWFJDHZYP6JI33K6FTDPVH4M24IPQXMWQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.065684383 (50h53m54.105257503s) [noise=true , address=45.91.116.12:4227 , piece_id=LTRRHRUI27RT7EAARZ6URA4PLNTQ4KBYJSUFDRZAPPPO5DWXRIPA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.074083004 (50h53m54.113656124s) [noise=true , address=45.91.116.6:4974 , piece_id=45JH67CWGKWHQG3J4TVSPUKVQKNEB3LD6NX5MHTJXDSEF2HSFQNA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.085711967 (50h53m54.125285087s) [noise=true , address=77.90.31.10:4518 , piece_id=NPYH4EAUAKKXSGPAP42NJEZONTN7S3GSMJWTGH3UQNMZTQISUVBQ, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089181967 (50h53m54.128755087s) [noise=true , address=77.90.31.21:4879 , piece_id=NXWVCXEWQMRQ6F3RZLOL6NQX7EBBFHCQZ3HGNBRTDCX4S2HIJPSA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: <nil> Sep 8 16:22:18.089206824 (50h53m54.128779944s) [seg_pos={0 0}, upload=44720]: Segment reached optimal threshold of 13 pieces. Sep 8 16:22:18.089320151 (50h53m54.128893271s) [noise=true , address=77.90.31.19:4841 , piece_id=RZDAN4AYQICOBXUWMTVAJP3R4TYXO7KW3YZFJJJ3HF5KWCDOH7YA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089376610 (50h53m54.12894973s) [noise=true , address=145.79.227.15:4876 , piece_id=HQTP4VLWOBF7LVORWCITW4YC2UDZFHXRMRAXYODZTOOEI2S3WXXA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089467584 (50h53m54.129040704s) [noise=true , address=145.79.227.13:4802 , piece_id=RQ6OD4MCZ7AZKBJFL3XHQYF7GJ7HIJSC7TTE2UFSXEFQ3KNBOGJA, seg_pos={0 0}, upload=44720]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4): protocol: expected piece hash; context canceled; EOF Sep 8 16:22:18.089520996 (50h53m54.129094116s) [seg_pos={0 0}, upload=44720]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:22:18.089622591 (50h53m54.129195711s) [upload=44720]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:22:18.089629243 (50h53m54.129202373s) [upload=44720]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:22:18.473990726 (50h53m54.513563846s) [upload=44720]: Done uploading segments... Sep 8 16:22:18.474015564 (50h53m54.513588684s) [upload=44720]: Upload finished. err: <nil> Duration: 30.002189068s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......X..XXX..........X....................................] duration: 6.343208854s FAIL (2025-09-08 16:22:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.343236868s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [........X..XXX..........X...................................] duration: 6.458436588s FAIL (2025-09-08 16:21:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.458462469s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............X......................X...X..................] duration: 6.828481666s FAIL (2025-09-08 16:21:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:08:24.413421670 (50h40m0.4529948s) [upload=44521]: Starting upload Sep 8 16:08:24.413511181 (50h40m0.453084301s) [upload=44521]: Done starting upload Sep 8 16:08:24.413613276 (50h40m0.453186397s) [upload=44521]: Uploading segments... Sep 8 16:08:24.413756059 (50h40m0.45332918s) [upload=44521]: Got next segment. Inline: false Sep 8 16:08:24.413786748 (50h40m0.453359858s) [upload=44521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:08:24.413791226 (50h40m0.453364347s) [upload=44521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:08:24.688933263 (50h40m0.728506383s) [seg_pos={0 0}, upload=44521]: Begin upload segment... Sep 8 16:08:24.688998427 (50h40m0.728571598s) [seg_pos={0 0}, upload=44521]: Done begin upload segment. Sep 8 16:08:24.689039316 (50h40m0.728612436s) [noise=true , address=77.90.31.14:4662 , piece_id=XWP5AWV3IPPDI2KXSRIUO5PV733NRUGUW5GSQEXQO33DKDD2URFA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.689048103 (50h40m0.728621223s) [noise=true , address=77.90.31.15:4699 , piece_id=YK7QSHB244GAUDWCLFBGRDRTBAXKMS3PCWMKHVCDJGL7NHX5U4AA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.689552779 (50h40m0.729125899s) [noise=true , address=77.90.31.19:4843 , piece_id=KCEL6V3JZRJQ6A7NOZQ4P2KONTQAJNTCJNF4NIDBMSH33GPJJQCA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.689868502 (50h40m0.729441623s) [noise=true , address=145.79.227.3:4151 , piece_id=N5X5OBQVN5TWYUUACQ46TJQU3WAQB4OONRNIZTMUTDM42ALZNMHQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690135944 (50h40m0.729709064s) [noise=true , address=45.91.116.12:4225 , piece_id=5TK5AFJ37LD6APCYI77U7BC5ZSJFJH2S4QYKMKGAT3IPS3S7UWFQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690473119 (50h40m0.730046249s) [noise=true , address=77.90.31.10:4518 , piece_id=U34YLH6YMKT4EMX22EY2FXUMTDX4FN6SKQ2PTQT7YQEVURUSUZDA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690771941 (50h40m0.730345061s) [noise=true , address=45.91.116.13:4153 , piece_id=NTNPJURMYQWIXPG27T22CIHMB4YSYXBWYFUOZ2FGNQ5SXTIGPKZQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690987404 (50h40m0.730560524s) [noise=true , address=45.91.116.7:4011 , piece_id=YPFV6N5TS5FXAJW6VZGRNF55E5RSA5BKSVJWC7H2SJUBKPVH6QVA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691202235 (50h40m0.730775345s) [noise=true , address=45.91.116.11:4119 , piece_id=3FQTKSX5MNAIQPXHHIDQGCOHJ3WBUJXO3ICSBB2JRRDRXWYXMSVA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691411375 (50h40m0.730984495s) [noise=true , address=45.91.116.14:4191 , piece_id=WYCYDMHK4DOR2SDVBZURVSM4WDKIOWDUJG7BS5E4GAXNDOXR57XA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691598322 (50h40m0.731171442s) [noise=true , address=77.90.31.13:4627 , piece_id=GWVGPZYZJAFETOYNIOW27F3YEIUTBO6VJTPRDW4SBNW6HWJTURUA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691807092 (50h40m0.731380212s) [noise=true , address=77.90.31.21:4879 , piece_id=H23WENKOHGIJKB3PQG5X7QCDCAIA4SO4FHRW7ZOB75EYDSJVQN7Q, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692009178 (50h40m0.731582298s) [noise=true , address=45.91.116.10:4082 , piece_id=IVQ6YJQGVK7QCXW64HL4VXZF5IKKXRPJ5A22BMHULFAY4IT3LAYA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692211446 (50h40m0.731784566s) [noise=true , address=45.91.116.11:4118 , piece_id=TF5UDY4CJ6YRLH5IUEXQ7OYBTSFG4YVH4P7DD6R344VGPT7UCO3A, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692403824 (50h40m0.731976944s) [noise=true , address=45.91.116.6:4973 , piece_id=735AY7MZQRVG5Q4XMPT5GBTKGYCDX6NL34RCNIOMTIAUHNBOCWQA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692591674 (50h40m0.732164794s) [noise=true , address=45.91.116.5:4299 , piece_id=B7XEFUQTEH7PSN6XSM5IWKJHAAA23W2U4JTCFZ3C5UAFG7X42SHQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:26.488618932 (50h40m2.528192062s) [upload=44521]: Next returned nil segment Sep 8 16:08:26.488638038 (50h40m2.528211158s) [upload=44521]: Waiting for error group managing segments... Sep 8 16:08:26.782849935 (50h40m2.822423055s) [noise=true , address=45.91.116.6:4973 , piece_id=735AY7MZQRVG5Q4XMPT5GBTKGYCDX6NL34RCNIOMTIAUHNBOCWQA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:26.966056559 (50h40m3.005629679s) [noise=true , address=77.90.31.14:4662 , piece_id=XWP5AWV3IPPDI2KXSRIUO5PV733NRUGUW5GSQEXQO33DKDD2URFA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.035362168 (50h40m3.074935288s) [noise=true , address=45.91.116.11:4118 , piece_id=TF5UDY4CJ6YRLH5IUEXQ7OYBTSFG4YVH4P7DD6R344VGPT7UCO3A, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.097946837 (50h40m3.137519957s) [noise=true , address=145.79.227.3:4151 , piece_id=N5X5OBQVN5TWYUUACQ46TJQU3WAQB4OONRNIZTMUTDM42ALZNMHQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.160333006 (50h40m3.199906126s) [noise=true , address=77.90.31.10:4518 , piece_id=U34YLH6YMKT4EMX22EY2FXUMTDX4FN6SKQ2PTQT7YQEVURUSUZDA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.250934906 (50h40m3.290508026s) [noise=true , address=45.91.116.12:4225 , piece_id=5TK5AFJ37LD6APCYI77U7BC5ZSJFJH2S4QYKMKGAT3IPS3S7UWFQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.257381642 (50h40m3.296954762s) [noise=true , address=45.91.116.11:4119 , piece_id=3FQTKSX5MNAIQPXHHIDQGCOHJ3WBUJXO3ICSBB2JRRDRXWYXMSVA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.298283861 (50h40m3.337856981s) [noise=true , address=77.90.31.15:4699 , piece_id=YK7QSHB244GAUDWCLFBGRDRTBAXKMS3PCWMKHVCDJGL7NHX5U4AA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.329488312 (50h40m3.369061432s) [noise=true , address=45.91.116.5:4299 , piece_id=B7XEFUQTEH7PSN6XSM5IWKJHAAA23W2U4JTCFZ3C5UAFG7X42SHQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.338479798 (50h40m3.378052908s) [noise=true , address=45.91.116.7:4011 , piece_id=YPFV6N5TS5FXAJW6VZGRNF55E5RSA5BKSVJWC7H2SJUBKPVH6QVA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.369023604 (50h40m3.408596724s) [noise=true , address=77.90.31.21:4879 , piece_id=H23WENKOHGIJKB3PQG5X7QCDCAIA4SO4FHRW7ZOB75EYDSJVQN7Q, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:28.027636399 (50h40m4.067209529s) [noise=true , address=77.90.31.19:4843 , piece_id=KCEL6V3JZRJQ6A7NOZQ4P2KONTQAJNTCJNF4NIDBMSH33GPJJQCA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:28.088958591 (50h40m4.128531721s) [noise=true , address=45.91.116.14:4191 , piece_id=WYCYDMHK4DOR2SDVBZURVSM4WDKIOWDUJG7BS5E4GAXNDOXR57XA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:28.088984380 (50h40m4.1285575s) [seg_pos={0 0}, upload=44521]: Segment reached optimal threshold of 13 pieces. Sep 8 16:08:28.089102657 (50h40m4.128675777s) [noise=true , address=77.90.31.13:4627 , piece_id=GWVGPZYZJAFETOYNIOW27F3YEIUTBO6VJTPRDW4SBNW6HWJTURUA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB): protocol: expected piece hash; context canceled; EOF Sep 8 16:08:28.089147282 (50h40m4.128720402s) [noise=true , address=45.91.116.13:4153 , piece_id=NTNPJURMYQWIXPG27T22CIHMB4YSYXBWYFUOZ2FGNQ5SXTIGPKZQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b): protocol: expected piece hash; context canceled; EOF Sep 8 16:08:28.089173873 (50h40m4.128746993s) [noise=true , address=45.91.116.10:4082 , piece_id=IVQ6YJQGVK7QCXW64HL4VXZF5IKKXRPJ5A22BMHULFAY4IT3LAYA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Sep 8 16:08:28.089283673 (50h40m4.128856793s) [seg_pos={0 0}, upload=44521]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:08:28.089409153 (50h40m4.128982283s) [upload=44521]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:08:28.089435003 (50h40m4.129008133s) [upload=44521]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:08:28.534343825 (50h40m4.573916945s) [upload=44521]: Done uploading segments... Sep 8 16:08:28.534364004 (50h40m4.573937124s) [upload=44521]: Upload finished. err: <nil> Duration: 6.828511444s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............X......................X...X.................] duration: 6.559437015s FAIL (2025-09-08 16:20:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:08:24.413421670 (50h40m0.4529948s) [upload=44521]: Starting upload Sep 8 16:08:24.413511181 (50h40m0.453084301s) [upload=44521]: Done starting upload Sep 8 16:08:24.413613276 (50h40m0.453186397s) [upload=44521]: Uploading segments... Sep 8 16:08:24.413756059 (50h40m0.45332918s) [upload=44521]: Got next segment. Inline: false Sep 8 16:08:24.413786748 (50h40m0.453359858s) [upload=44521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:08:24.413791226 (50h40m0.453364347s) [upload=44521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:08:24.688933263 (50h40m0.728506383s) [seg_pos={0 0}, upload=44521]: Begin upload segment... Sep 8 16:08:24.688998427 (50h40m0.728571598s) [seg_pos={0 0}, upload=44521]: Done begin upload segment. Sep 8 16:08:24.689039316 (50h40m0.728612436s) [noise=true , address=77.90.31.14:4662 , piece_id=XWP5AWV3IPPDI2KXSRIUO5PV733NRUGUW5GSQEXQO33DKDD2URFA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.689048103 (50h40m0.728621223s) [noise=true , address=77.90.31.15:4699 , piece_id=YK7QSHB244GAUDWCLFBGRDRTBAXKMS3PCWMKHVCDJGL7NHX5U4AA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.689552779 (50h40m0.729125899s) [noise=true , address=77.90.31.19:4843 , piece_id=KCEL6V3JZRJQ6A7NOZQ4P2KONTQAJNTCJNF4NIDBMSH33GPJJQCA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.689868502 (50h40m0.729441623s) [noise=true , address=145.79.227.3:4151 , piece_id=N5X5OBQVN5TWYUUACQ46TJQU3WAQB4OONRNIZTMUTDM42ALZNMHQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690135944 (50h40m0.729709064s) [noise=true , address=45.91.116.12:4225 , piece_id=5TK5AFJ37LD6APCYI77U7BC5ZSJFJH2S4QYKMKGAT3IPS3S7UWFQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690473119 (50h40m0.730046249s) [noise=true , address=77.90.31.10:4518 , piece_id=U34YLH6YMKT4EMX22EY2FXUMTDX4FN6SKQ2PTQT7YQEVURUSUZDA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690771941 (50h40m0.730345061s) [noise=true , address=45.91.116.13:4153 , piece_id=NTNPJURMYQWIXPG27T22CIHMB4YSYXBWYFUOZ2FGNQ5SXTIGPKZQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690987404 (50h40m0.730560524s) [noise=true , address=45.91.116.7:4011 , piece_id=YPFV6N5TS5FXAJW6VZGRNF55E5RSA5BKSVJWC7H2SJUBKPVH6QVA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691202235 (50h40m0.730775345s) [noise=true , address=45.91.116.11:4119 , piece_id=3FQTKSX5MNAIQPXHHIDQGCOHJ3WBUJXO3ICSBB2JRRDRXWYXMSVA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691411375 (50h40m0.730984495s) [noise=true , address=45.91.116.14:4191 , piece_id=WYCYDMHK4DOR2SDVBZURVSM4WDKIOWDUJG7BS5E4GAXNDOXR57XA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691598322 (50h40m0.731171442s) [noise=true , address=77.90.31.13:4627 , piece_id=GWVGPZYZJAFETOYNIOW27F3YEIUTBO6VJTPRDW4SBNW6HWJTURUA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691807092 (50h40m0.731380212s) [noise=true , address=77.90.31.21:4879 , piece_id=H23WENKOHGIJKB3PQG5X7QCDCAIA4SO4FHRW7ZOB75EYDSJVQN7Q, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692009178 (50h40m0.731582298s) [noise=true , address=45.91.116.10:4082 , piece_id=IVQ6YJQGVK7QCXW64HL4VXZF5IKKXRPJ5A22BMHULFAY4IT3LAYA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692211446 (50h40m0.731784566s) [noise=true , address=45.91.116.11:4118 , piece_id=TF5UDY4CJ6YRLH5IUEXQ7OYBTSFG4YVH4P7DD6R344VGPT7UCO3A, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692403824 (50h40m0.731976944s) [noise=true , address=45.91.116.6:4973 , piece_id=735AY7MZQRVG5Q4XMPT5GBTKGYCDX6NL34RCNIOMTIAUHNBOCWQA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692591674 (50h40m0.732164794s) [noise=true , address=45.91.116.5:4299 , piece_id=B7XEFUQTEH7PSN6XSM5IWKJHAAA23W2U4JTCFZ3C5UAFG7X42SHQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:26.488618932 (50h40m2.528192062s) [upload=44521]: Next returned nil segment Sep 8 16:08:26.488638038 (50h40m2.528211158s) [upload=44521]: Waiting for error group managing segments... Sep 8 16:08:26.782849935 (50h40m2.822423055s) [noise=true , address=45.91.116.6:4973 , piece_id=735AY7MZQRVG5Q4XMPT5GBTKGYCDX6NL34RCNIOMTIAUHNBOCWQA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:26.966056559 (50h40m3.005629679s) [noise=true , address=77.90.31.14:4662 , piece_id=XWP5AWV3IPPDI2KXSRIUO5PV733NRUGUW5GSQEXQO33DKDD2URFA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.035362168 (50h40m3.074935288s) [noise=true , address=45.91.116.11:4118 , piece_id=TF5UDY4CJ6YRLH5IUEXQ7OYBTSFG4YVH4P7DD6R344VGPT7UCO3A, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.097946837 (50h40m3.137519957s) [noise=true , address=145.79.227.3:4151 , piece_id=N5X5OBQVN5TWYUUACQ46TJQU3WAQB4OONRNIZTMUTDM42ALZNMHQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.160333006 (50h40m3.199906126s) [noise=true , address=77.90.31.10:4518 , piece_id=U34YLH6YMKT4EMX22EY2FXUMTDX4FN6SKQ2PTQT7YQEVURUSUZDA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.250934906 (50h40m3.290508026s) [noise=true , address=45.91.116.12:4225 , piece_id=5TK5AFJ37LD6APCYI77U7BC5ZSJFJH2S4QYKMKGAT3IPS3S7UWFQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.257381642 (50h40m3.296954762s) [noise=true , address=45.91.116.11:4119 , piece_id=3FQTKSX5MNAIQPXHHIDQGCOHJ3WBUJXO3ICSBB2JRRDRXWYXMSVA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.298283861 (50h40m3.337856981s) [noise=true , address=77.90.31.15:4699 , piece_id=YK7QSHB244GAUDWCLFBGRDRTBAXKMS3PCWMKHVCDJGL7NHX5U4AA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.329488312 (50h40m3.369061432s) [noise=true , address=45.91.116.5:4299 , piece_id=B7XEFUQTEH7PSN6XSM5IWKJHAAA23W2U4JTCFZ3C5UAFG7X42SHQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.338479798 (50h40m3.378052908s) [noise=true , address=45.91.116.7:4011 , piece_id=YPFV6N5TS5FXAJW6VZGRNF55E5RSA5BKSVJWC7H2SJUBKPVH6QVA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.369023604 (50h40m3.408596724s) [noise=true , address=77.90.31.21:4879 , piece_id=H23WENKOHGIJKB3PQG5X7QCDCAIA4SO4FHRW7ZOB75EYDSJVQN7Q, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:28.027636399 (50h40m4.067209529s) [noise=true , address=77.90.31.19:4843 , piece_id=KCEL6V3JZRJQ6A7NOZQ4P2KONTQAJNTCJNF4NIDBMSH33GPJJQCA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:28.088958591 (50h40m4.128531721s) [noise=true , address=45.91.116.14:4191 , piece_id=WYCYDMHK4DOR2SDVBZURVSM4WDKIOWDUJG7BS5E4GAXNDOXR57XA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:28.088984380 (50h40m4.1285575s) [seg_pos={0 0}, upload=44521]: Segment reached optimal threshold of 13 pieces. Sep 8 16:08:28.089102657 (50h40m4.128675777s) [noise=true , address=77.90.31.13:4627 , piece_id=GWVGPZYZJAFETOYNIOW27F3YEIUTBO6VJTPRDW4SBNW6HWJTURUA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB): protocol: expected piece hash; context canceled; EOF Sep 8 16:08:28.089147282 (50h40m4.128720402s) [noise=true , address=45.91.116.13:4153 , piece_id=NTNPJURMYQWIXPG27T22CIHMB4YSYXBWYFUOZ2FGNQ5SXTIGPKZQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b): protocol: expected piece hash; context canceled; EOF Sep 8 16:08:28.089173873 (50h40m4.128746993s) [noise=true , address=45.91.116.10:4082 , piece_id=IVQ6YJQGVK7QCXW64HL4VXZF5IKKXRPJ5A22BMHULFAY4IT3LAYA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Sep 8 16:08:28.089283673 (50h40m4.128856793s) [seg_pos={0 0}, upload=44521]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:08:28.089409153 (50h40m4.128982283s) [upload=44521]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:08:28.089435003 (50h40m4.129008133s) [upload=44521]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:08:28.534343825 (50h40m4.573916945s) [upload=44521]: Done uploading segments... Sep 8 16:08:28.534364004 (50h40m4.573937124s) [upload=44521]: Upload finished. err: <nil> Duration: 6.559463936s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.........X..XXX..........X..................................] duration: 6.676913731s FAIL (2025-09-08 16:20:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.676940232s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..........X..XXX..........X.................................] duration: 6.521698805s FAIL (2025-09-08 16:20:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.521728902s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................X......................X...X................] duration: 7.088681387s FAIL (2025-09-08 16:19:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:08:24.413421670 (50h40m0.4529948s) [upload=44521]: Starting upload Sep 8 16:08:24.413511181 (50h40m0.453084301s) [upload=44521]: Done starting upload Sep 8 16:08:24.413613276 (50h40m0.453186397s) [upload=44521]: Uploading segments... Sep 8 16:08:24.413756059 (50h40m0.45332918s) [upload=44521]: Got next segment. Inline: false Sep 8 16:08:24.413786748 (50h40m0.453359858s) [upload=44521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:08:24.413791226 (50h40m0.453364347s) [upload=44521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:08:24.688933263 (50h40m0.728506383s) [seg_pos={0 0}, upload=44521]: Begin upload segment... Sep 8 16:08:24.688998427 (50h40m0.728571598s) [seg_pos={0 0}, upload=44521]: Done begin upload segment. Sep 8 16:08:24.689039316 (50h40m0.728612436s) [noise=true , address=77.90.31.14:4662 , piece_id=XWP5AWV3IPPDI2KXSRIUO5PV733NRUGUW5GSQEXQO33DKDD2URFA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.689048103 (50h40m0.728621223s) [noise=true , address=77.90.31.15:4699 , piece_id=YK7QSHB244GAUDWCLFBGRDRTBAXKMS3PCWMKHVCDJGL7NHX5U4AA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.689552779 (50h40m0.729125899s) [noise=true , address=77.90.31.19:4843 , piece_id=KCEL6V3JZRJQ6A7NOZQ4P2KONTQAJNTCJNF4NIDBMSH33GPJJQCA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.689868502 (50h40m0.729441623s) [noise=true , address=145.79.227.3:4151 , piece_id=N5X5OBQVN5TWYUUACQ46TJQU3WAQB4OONRNIZTMUTDM42ALZNMHQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690135944 (50h40m0.729709064s) [noise=true , address=45.91.116.12:4225 , piece_id=5TK5AFJ37LD6APCYI77U7BC5ZSJFJH2S4QYKMKGAT3IPS3S7UWFQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690473119 (50h40m0.730046249s) [noise=true , address=77.90.31.10:4518 , piece_id=U34YLH6YMKT4EMX22EY2FXUMTDX4FN6SKQ2PTQT7YQEVURUSUZDA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690771941 (50h40m0.730345061s) [noise=true , address=45.91.116.13:4153 , piece_id=NTNPJURMYQWIXPG27T22CIHMB4YSYXBWYFUOZ2FGNQ5SXTIGPKZQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690987404 (50h40m0.730560524s) [noise=true , address=45.91.116.7:4011 , piece_id=YPFV6N5TS5FXAJW6VZGRNF55E5RSA5BKSVJWC7H2SJUBKPVH6QVA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691202235 (50h40m0.730775345s) [noise=true , address=45.91.116.11:4119 , piece_id=3FQTKSX5MNAIQPXHHIDQGCOHJ3WBUJXO3ICSBB2JRRDRXWYXMSVA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691411375 (50h40m0.730984495s) [noise=true , address=45.91.116.14:4191 , piece_id=WYCYDMHK4DOR2SDVBZURVSM4WDKIOWDUJG7BS5E4GAXNDOXR57XA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691598322 (50h40m0.731171442s) [noise=true , address=77.90.31.13:4627 , piece_id=GWVGPZYZJAFETOYNIOW27F3YEIUTBO6VJTPRDW4SBNW6HWJTURUA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691807092 (50h40m0.731380212s) [noise=true , address=77.90.31.21:4879 , piece_id=H23WENKOHGIJKB3PQG5X7QCDCAIA4SO4FHRW7ZOB75EYDSJVQN7Q, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692009178 (50h40m0.731582298s) [noise=true , address=45.91.116.10:4082 , piece_id=IVQ6YJQGVK7QCXW64HL4VXZF5IKKXRPJ5A22BMHULFAY4IT3LAYA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692211446 (50h40m0.731784566s) [noise=true , address=45.91.116.11:4118 , piece_id=TF5UDY4CJ6YRLH5IUEXQ7OYBTSFG4YVH4P7DD6R344VGPT7UCO3A, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692403824 (50h40m0.731976944s) [noise=true , address=45.91.116.6:4973 , piece_id=735AY7MZQRVG5Q4XMPT5GBTKGYCDX6NL34RCNIOMTIAUHNBOCWQA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692591674 (50h40m0.732164794s) [noise=true , address=45.91.116.5:4299 , piece_id=B7XEFUQTEH7PSN6XSM5IWKJHAAA23W2U4JTCFZ3C5UAFG7X42SHQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:26.488618932 (50h40m2.528192062s) [upload=44521]: Next returned nil segment Sep 8 16:08:26.488638038 (50h40m2.528211158s) [upload=44521]: Waiting for error group managing segments... Sep 8 16:08:26.782849935 (50h40m2.822423055s) [noise=true , address=45.91.116.6:4973 , piece_id=735AY7MZQRVG5Q4XMPT5GBTKGYCDX6NL34RCNIOMTIAUHNBOCWQA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:26.966056559 (50h40m3.005629679s) [noise=true , address=77.90.31.14:4662 , piece_id=XWP5AWV3IPPDI2KXSRIUO5PV733NRUGUW5GSQEXQO33DKDD2URFA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.035362168 (50h40m3.074935288s) [noise=true , address=45.91.116.11:4118 , piece_id=TF5UDY4CJ6YRLH5IUEXQ7OYBTSFG4YVH4P7DD6R344VGPT7UCO3A, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.097946837 (50h40m3.137519957s) [noise=true , address=145.79.227.3:4151 , piece_id=N5X5OBQVN5TWYUUACQ46TJQU3WAQB4OONRNIZTMUTDM42ALZNMHQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.160333006 (50h40m3.199906126s) [noise=true , address=77.90.31.10:4518 , piece_id=U34YLH6YMKT4EMX22EY2FXUMTDX4FN6SKQ2PTQT7YQEVURUSUZDA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.250934906 (50h40m3.290508026s) [noise=true , address=45.91.116.12:4225 , piece_id=5TK5AFJ37LD6APCYI77U7BC5ZSJFJH2S4QYKMKGAT3IPS3S7UWFQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.257381642 (50h40m3.296954762s) [noise=true , address=45.91.116.11:4119 , piece_id=3FQTKSX5MNAIQPXHHIDQGCOHJ3WBUJXO3ICSBB2JRRDRXWYXMSVA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.298283861 (50h40m3.337856981s) [noise=true , address=77.90.31.15:4699 , piece_id=YK7QSHB244GAUDWCLFBGRDRTBAXKMS3PCWMKHVCDJGL7NHX5U4AA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.329488312 (50h40m3.369061432s) [noise=true , address=45.91.116.5:4299 , piece_id=B7XEFUQTEH7PSN6XSM5IWKJHAAA23W2U4JTCFZ3C5UAFG7X42SHQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.338479798 (50h40m3.378052908s) [noise=true , address=45.91.116.7:4011 , piece_id=YPFV6N5TS5FXAJW6VZGRNF55E5RSA5BKSVJWC7H2SJUBKPVH6QVA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.369023604 (50h40m3.408596724s) [noise=true , address=77.90.31.21:4879 , piece_id=H23WENKOHGIJKB3PQG5X7QCDCAIA4SO4FHRW7ZOB75EYDSJVQN7Q, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:28.027636399 (50h40m4.067209529s) [noise=true , address=77.90.31.19:4843 , piece_id=KCEL6V3JZRJQ6A7NOZQ4P2KONTQAJNTCJNF4NIDBMSH33GPJJQCA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:28.088958591 (50h40m4.128531721s) [noise=true , address=45.91.116.14:4191 , piece_id=WYCYDMHK4DOR2SDVBZURVSM4WDKIOWDUJG7BS5E4GAXNDOXR57XA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:28.088984380 (50h40m4.1285575s) [seg_pos={0 0}, upload=44521]: Segment reached optimal threshold of 13 pieces. Sep 8 16:08:28.089102657 (50h40m4.128675777s) [noise=true , address=77.90.31.13:4627 , piece_id=GWVGPZYZJAFETOYNIOW27F3YEIUTBO6VJTPRDW4SBNW6HWJTURUA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB): protocol: expected piece hash; context canceled; EOF Sep 8 16:08:28.089147282 (50h40m4.128720402s) [noise=true , address=45.91.116.13:4153 , piece_id=NTNPJURMYQWIXPG27T22CIHMB4YSYXBWYFUOZ2FGNQ5SXTIGPKZQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b): protocol: expected piece hash; context canceled; EOF Sep 8 16:08:28.089173873 (50h40m4.128746993s) [noise=true , address=45.91.116.10:4082 , piece_id=IVQ6YJQGVK7QCXW64HL4VXZF5IKKXRPJ5A22BMHULFAY4IT3LAYA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Sep 8 16:08:28.089283673 (50h40m4.128856793s) [seg_pos={0 0}, upload=44521]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:08:28.089409153 (50h40m4.128982283s) [upload=44521]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:08:28.089435003 (50h40m4.129008133s) [upload=44521]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:08:28.534343825 (50h40m4.573916945s) [upload=44521]: Done uploading segments... Sep 8 16:08:28.534364004 (50h40m4.573937124s) [upload=44521]: Upload finished. err: <nil> Duration: 7.08871359s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...........X..XXX..........X................................] duration: 6.409721419s FAIL (2025-09-08 16:19:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.409744422s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................X......................X...X...............] duration: 7.61439208s FAIL (2025-09-08 16:18:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:08:24.413421670 (50h40m0.4529948s) [upload=44521]: Starting upload Sep 8 16:08:24.413511181 (50h40m0.453084301s) [upload=44521]: Done starting upload Sep 8 16:08:24.413613276 (50h40m0.453186397s) [upload=44521]: Uploading segments... Sep 8 16:08:24.413756059 (50h40m0.45332918s) [upload=44521]: Got next segment. Inline: false Sep 8 16:08:24.413786748 (50h40m0.453359858s) [upload=44521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:08:24.413791226 (50h40m0.453364347s) [upload=44521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:08:24.688933263 (50h40m0.728506383s) [seg_pos={0 0}, upload=44521]: Begin upload segment... Sep 8 16:08:24.688998427 (50h40m0.728571598s) [seg_pos={0 0}, upload=44521]: Done begin upload segment. Sep 8 16:08:24.689039316 (50h40m0.728612436s) [noise=true , address=77.90.31.14:4662 , piece_id=XWP5AWV3IPPDI2KXSRIUO5PV733NRUGUW5GSQEXQO33DKDD2URFA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.689048103 (50h40m0.728621223s) [noise=true , address=77.90.31.15:4699 , piece_id=YK7QSHB244GAUDWCLFBGRDRTBAXKMS3PCWMKHVCDJGL7NHX5U4AA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.689552779 (50h40m0.729125899s) [noise=true , address=77.90.31.19:4843 , piece_id=KCEL6V3JZRJQ6A7NOZQ4P2KONTQAJNTCJNF4NIDBMSH33GPJJQCA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.689868502 (50h40m0.729441623s) [noise=true , address=145.79.227.3:4151 , piece_id=N5X5OBQVN5TWYUUACQ46TJQU3WAQB4OONRNIZTMUTDM42ALZNMHQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690135944 (50h40m0.729709064s) [noise=true , address=45.91.116.12:4225 , piece_id=5TK5AFJ37LD6APCYI77U7BC5ZSJFJH2S4QYKMKGAT3IPS3S7UWFQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690473119 (50h40m0.730046249s) [noise=true , address=77.90.31.10:4518 , piece_id=U34YLH6YMKT4EMX22EY2FXUMTDX4FN6SKQ2PTQT7YQEVURUSUZDA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690771941 (50h40m0.730345061s) [noise=true , address=45.91.116.13:4153 , piece_id=NTNPJURMYQWIXPG27T22CIHMB4YSYXBWYFUOZ2FGNQ5SXTIGPKZQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690987404 (50h40m0.730560524s) [noise=true , address=45.91.116.7:4011 , piece_id=YPFV6N5TS5FXAJW6VZGRNF55E5RSA5BKSVJWC7H2SJUBKPVH6QVA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691202235 (50h40m0.730775345s) [noise=true , address=45.91.116.11:4119 , piece_id=3FQTKSX5MNAIQPXHHIDQGCOHJ3WBUJXO3ICSBB2JRRDRXWYXMSVA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691411375 (50h40m0.730984495s) [noise=true , address=45.91.116.14:4191 , piece_id=WYCYDMHK4DOR2SDVBZURVSM4WDKIOWDUJG7BS5E4GAXNDOXR57XA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691598322 (50h40m0.731171442s) [noise=true , address=77.90.31.13:4627 , piece_id=GWVGPZYZJAFETOYNIOW27F3YEIUTBO6VJTPRDW4SBNW6HWJTURUA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691807092 (50h40m0.731380212s) [noise=true , address=77.90.31.21:4879 , piece_id=H23WENKOHGIJKB3PQG5X7QCDCAIA4SO4FHRW7ZOB75EYDSJVQN7Q, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692009178 (50h40m0.731582298s) [noise=true , address=45.91.116.10:4082 , piece_id=IVQ6YJQGVK7QCXW64HL4VXZF5IKKXRPJ5A22BMHULFAY4IT3LAYA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692211446 (50h40m0.731784566s) [noise=true , address=45.91.116.11:4118 , piece_id=TF5UDY4CJ6YRLH5IUEXQ7OYBTSFG4YVH4P7DD6R344VGPT7UCO3A, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692403824 (50h40m0.731976944s) [noise=true , address=45.91.116.6:4973 , piece_id=735AY7MZQRVG5Q4XMPT5GBTKGYCDX6NL34RCNIOMTIAUHNBOCWQA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692591674 (50h40m0.732164794s) [noise=true , address=45.91.116.5:4299 , piece_id=B7XEFUQTEH7PSN6XSM5IWKJHAAA23W2U4JTCFZ3C5UAFG7X42SHQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:26.488618932 (50h40m2.528192062s) [upload=44521]: Next returned nil segment Sep 8 16:08:26.488638038 (50h40m2.528211158s) [upload=44521]: Waiting for error group managing segments... Sep 8 16:08:26.782849935 (50h40m2.822423055s) [noise=true , address=45.91.116.6:4973 , piece_id=735AY7MZQRVG5Q4XMPT5GBTKGYCDX6NL34RCNIOMTIAUHNBOCWQA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:26.966056559 (50h40m3.005629679s) [noise=true , address=77.90.31.14:4662 , piece_id=XWP5AWV3IPPDI2KXSRIUO5PV733NRUGUW5GSQEXQO33DKDD2URFA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.035362168 (50h40m3.074935288s) [noise=true , address=45.91.116.11:4118 , piece_id=TF5UDY4CJ6YRLH5IUEXQ7OYBTSFG4YVH4P7DD6R344VGPT7UCO3A, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.097946837 (50h40m3.137519957s) [noise=true , address=145.79.227.3:4151 , piece_id=N5X5OBQVN5TWYUUACQ46TJQU3WAQB4OONRNIZTMUTDM42ALZNMHQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.160333006 (50h40m3.199906126s) [noise=true , address=77.90.31.10:4518 , piece_id=U34YLH6YMKT4EMX22EY2FXUMTDX4FN6SKQ2PTQT7YQEVURUSUZDA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.250934906 (50h40m3.290508026s) [noise=true , address=45.91.116.12:4225 , piece_id=5TK5AFJ37LD6APCYI77U7BC5ZSJFJH2S4QYKMKGAT3IPS3S7UWFQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.257381642 (50h40m3.296954762s) [noise=true , address=45.91.116.11:4119 , piece_id=3FQTKSX5MNAIQPXHHIDQGCOHJ3WBUJXO3ICSBB2JRRDRXWYXMSVA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.298283861 (50h40m3.337856981s) [noise=true , address=77.90.31.15:4699 , piece_id=YK7QSHB244GAUDWCLFBGRDRTBAXKMS3PCWMKHVCDJGL7NHX5U4AA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.329488312 (50h40m3.369061432s) [noise=true , address=45.91.116.5:4299 , piece_id=B7XEFUQTEH7PSN6XSM5IWKJHAAA23W2U4JTCFZ3C5UAFG7X42SHQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.338479798 (50h40m3.378052908s) [noise=true , address=45.91.116.7:4011 , piece_id=YPFV6N5TS5FXAJW6VZGRNF55E5RSA5BKSVJWC7H2SJUBKPVH6QVA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.369023604 (50h40m3.408596724s) [noise=true , address=77.90.31.21:4879 , piece_id=H23WENKOHGIJKB3PQG5X7QCDCAIA4SO4FHRW7ZOB75EYDSJVQN7Q, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:28.027636399 (50h40m4.067209529s) [noise=true , address=77.90.31.19:4843 , piece_id=KCEL6V3JZRJQ6A7NOZQ4P2KONTQAJNTCJNF4NIDBMSH33GPJJQCA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:28.088958591 (50h40m4.128531721s) [noise=true , address=45.91.116.14:4191 , piece_id=WYCYDMHK4DOR2SDVBZURVSM4WDKIOWDUJG7BS5E4GAXNDOXR57XA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:28.088984380 (50h40m4.1285575s) [seg_pos={0 0}, upload=44521]: Segment reached optimal threshold of 13 pieces. Sep 8 16:08:28.089102657 (50h40m4.128675777s) [noise=true , address=77.90.31.13:4627 , piece_id=GWVGPZYZJAFETOYNIOW27F3YEIUTBO6VJTPRDW4SBNW6HWJTURUA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB): protocol: expected piece hash; context canceled; EOF Sep 8 16:08:28.089147282 (50h40m4.128720402s) [noise=true , address=45.91.116.13:4153 , piece_id=NTNPJURMYQWIXPG27T22CIHMB4YSYXBWYFUOZ2FGNQ5SXTIGPKZQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b): protocol: expected piece hash; context canceled; EOF Sep 8 16:08:28.089173873 (50h40m4.128746993s) [noise=true , address=45.91.116.10:4082 , piece_id=IVQ6YJQGVK7QCXW64HL4VXZF5IKKXRPJ5A22BMHULFAY4IT3LAYA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Sep 8 16:08:28.089283673 (50h40m4.128856793s) [seg_pos={0 0}, upload=44521]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:08:28.089409153 (50h40m4.128982283s) [upload=44521]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:08:28.089435003 (50h40m4.129008133s) [upload=44521]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:08:28.534343825 (50h40m4.573916945s) [upload=44521]: Done uploading segments... Sep 8 16:08:28.534364004 (50h40m4.573937124s) [upload=44521]: Upload finished. err: <nil> Duration: 7.614423351s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................X......................X...X..............] duration: 6.019877285s FAIL (2025-09-08 16:18:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:08:24.413421670 (50h40m0.4529948s) [upload=44521]: Starting upload Sep 8 16:08:24.413511181 (50h40m0.453084301s) [upload=44521]: Done starting upload Sep 8 16:08:24.413613276 (50h40m0.453186397s) [upload=44521]: Uploading segments... Sep 8 16:08:24.413756059 (50h40m0.45332918s) [upload=44521]: Got next segment. Inline: false Sep 8 16:08:24.413786748 (50h40m0.453359858s) [upload=44521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:08:24.413791226 (50h40m0.453364347s) [upload=44521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:08:24.688933263 (50h40m0.728506383s) [seg_pos={0 0}, upload=44521]: Begin upload segment... Sep 8 16:08:24.688998427 (50h40m0.728571598s) [seg_pos={0 0}, upload=44521]: Done begin upload segment. Sep 8 16:08:24.689039316 (50h40m0.728612436s) [noise=true , address=77.90.31.14:4662 , piece_id=XWP5AWV3IPPDI2KXSRIUO5PV733NRUGUW5GSQEXQO33DKDD2URFA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.689048103 (50h40m0.728621223s) [noise=true , address=77.90.31.15:4699 , piece_id=YK7QSHB244GAUDWCLFBGRDRTBAXKMS3PCWMKHVCDJGL7NHX5U4AA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.689552779 (50h40m0.729125899s) [noise=true , address=77.90.31.19:4843 , piece_id=KCEL6V3JZRJQ6A7NOZQ4P2KONTQAJNTCJNF4NIDBMSH33GPJJQCA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.689868502 (50h40m0.729441623s) [noise=true , address=145.79.227.3:4151 , piece_id=N5X5OBQVN5TWYUUACQ46TJQU3WAQB4OONRNIZTMUTDM42ALZNMHQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690135944 (50h40m0.729709064s) [noise=true , address=45.91.116.12:4225 , piece_id=5TK5AFJ37LD6APCYI77U7BC5ZSJFJH2S4QYKMKGAT3IPS3S7UWFQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690473119 (50h40m0.730046249s) [noise=true , address=77.90.31.10:4518 , piece_id=U34YLH6YMKT4EMX22EY2FXUMTDX4FN6SKQ2PTQT7YQEVURUSUZDA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690771941 (50h40m0.730345061s) [noise=true , address=45.91.116.13:4153 , piece_id=NTNPJURMYQWIXPG27T22CIHMB4YSYXBWYFUOZ2FGNQ5SXTIGPKZQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690987404 (50h40m0.730560524s) [noise=true , address=45.91.116.7:4011 , piece_id=YPFV6N5TS5FXAJW6VZGRNF55E5RSA5BKSVJWC7H2SJUBKPVH6QVA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691202235 (50h40m0.730775345s) [noise=true , address=45.91.116.11:4119 , piece_id=3FQTKSX5MNAIQPXHHIDQGCOHJ3WBUJXO3ICSBB2JRRDRXWYXMSVA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691411375 (50h40m0.730984495s) [noise=true , address=45.91.116.14:4191 , piece_id=WYCYDMHK4DOR2SDVBZURVSM4WDKIOWDUJG7BS5E4GAXNDOXR57XA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691598322 (50h40m0.731171442s) [noise=true , address=77.90.31.13:4627 , piece_id=GWVGPZYZJAFETOYNIOW27F3YEIUTBO6VJTPRDW4SBNW6HWJTURUA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691807092 (50h40m0.731380212s) [noise=true , address=77.90.31.21:4879 , piece_id=H23WENKOHGIJKB3PQG5X7QCDCAIA4SO4FHRW7ZOB75EYDSJVQN7Q, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692009178 (50h40m0.731582298s) [noise=true , address=45.91.116.10:4082 , piece_id=IVQ6YJQGVK7QCXW64HL4VXZF5IKKXRPJ5A22BMHULFAY4IT3LAYA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692211446 (50h40m0.731784566s) [noise=true , address=45.91.116.11:4118 , piece_id=TF5UDY4CJ6YRLH5IUEXQ7OYBTSFG4YVH4P7DD6R344VGPT7UCO3A, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692403824 (50h40m0.731976944s) [noise=true , address=45.91.116.6:4973 , piece_id=735AY7MZQRVG5Q4XMPT5GBTKGYCDX6NL34RCNIOMTIAUHNBOCWQA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692591674 (50h40m0.732164794s) [noise=true , address=45.91.116.5:4299 , piece_id=B7XEFUQTEH7PSN6XSM5IWKJHAAA23W2U4JTCFZ3C5UAFG7X42SHQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:26.488618932 (50h40m2.528192062s) [upload=44521]: Next returned nil segment Sep 8 16:08:26.488638038 (50h40m2.528211158s) [upload=44521]: Waiting for error group managing segments... Sep 8 16:08:26.782849935 (50h40m2.822423055s) [noise=true , address=45.91.116.6:4973 , piece_id=735AY7MZQRVG5Q4XMPT5GBTKGYCDX6NL34RCNIOMTIAUHNBOCWQA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:26.966056559 (50h40m3.005629679s) [noise=true , address=77.90.31.14:4662 , piece_id=XWP5AWV3IPPDI2KXSRIUO5PV733NRUGUW5GSQEXQO33DKDD2URFA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.035362168 (50h40m3.074935288s) [noise=true , address=45.91.116.11:4118 , piece_id=TF5UDY4CJ6YRLH5IUEXQ7OYBTSFG4YVH4P7DD6R344VGPT7UCO3A, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.097946837 (50h40m3.137519957s) [noise=true , address=145.79.227.3:4151 , piece_id=N5X5OBQVN5TWYUUACQ46TJQU3WAQB4OONRNIZTMUTDM42ALZNMHQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.160333006 (50h40m3.199906126s) [noise=true , address=77.90.31.10:4518 , piece_id=U34YLH6YMKT4EMX22EY2FXUMTDX4FN6SKQ2PTQT7YQEVURUSUZDA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.250934906 (50h40m3.290508026s) [noise=true , address=45.91.116.12:4225 , piece_id=5TK5AFJ37LD6APCYI77U7BC5ZSJFJH2S4QYKMKGAT3IPS3S7UWFQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.257381642 (50h40m3.296954762s) [noise=true , address=45.91.116.11:4119 , piece_id=3FQTKSX5MNAIQPXHHIDQGCOHJ3WBUJXO3ICSBB2JRRDRXWYXMSVA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.298283861 (50h40m3.337856981s) [noise=true , address=77.90.31.15:4699 , piece_id=YK7QSHB244GAUDWCLFBGRDRTBAXKMS3PCWMKHVCDJGL7NHX5U4AA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.329488312 (50h40m3.369061432s) [noise=true , address=45.91.116.5:4299 , piece_id=B7XEFUQTEH7PSN6XSM5IWKJHAAA23W2U4JTCFZ3C5UAFG7X42SHQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.338479798 (50h40m3.378052908s) [noise=true , address=45.91.116.7:4011 , piece_id=YPFV6N5TS5FXAJW6VZGRNF55E5RSA5BKSVJWC7H2SJUBKPVH6QVA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.369023604 (50h40m3.408596724s) [noise=true , address=77.90.31.21:4879 , piece_id=H23WENKOHGIJKB3PQG5X7QCDCAIA4SO4FHRW7ZOB75EYDSJVQN7Q, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:28.027636399 (50h40m4.067209529s) [noise=true , address=77.90.31.19:4843 , piece_id=KCEL6V3JZRJQ6A7NOZQ4P2KONTQAJNTCJNF4NIDBMSH33GPJJQCA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:28.088958591 (50h40m4.128531721s) [noise=true , address=45.91.116.14:4191 , piece_id=WYCYDMHK4DOR2SDVBZURVSM4WDKIOWDUJG7BS5E4GAXNDOXR57XA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:28.088984380 (50h40m4.1285575s) [seg_pos={0 0}, upload=44521]: Segment reached optimal threshold of 13 pieces. Sep 8 16:08:28.089102657 (50h40m4.128675777s) [noise=true , address=77.90.31.13:4627 , piece_id=GWVGPZYZJAFETOYNIOW27F3YEIUTBO6VJTPRDW4SBNW6HWJTURUA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB): protocol: expected piece hash; context canceled; EOF Sep 8 16:08:28.089147282 (50h40m4.128720402s) [noise=true , address=45.91.116.13:4153 , piece_id=NTNPJURMYQWIXPG27T22CIHMB4YSYXBWYFUOZ2FGNQ5SXTIGPKZQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b): protocol: expected piece hash; context canceled; EOF Sep 8 16:08:28.089173873 (50h40m4.128746993s) [noise=true , address=45.91.116.10:4082 , piece_id=IVQ6YJQGVK7QCXW64HL4VXZF5IKKXRPJ5A22BMHULFAY4IT3LAYA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Sep 8 16:08:28.089283673 (50h40m4.128856793s) [seg_pos={0 0}, upload=44521]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:08:28.089409153 (50h40m4.128982283s) [upload=44521]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:08:28.089435003 (50h40m4.129008133s) [upload=44521]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:08:28.534343825 (50h40m4.573916945s) [upload=44521]: Done uploading segments... Sep 8 16:08:28.534364004 (50h40m4.573937124s) [upload=44521]: Upload finished. err: <nil> Duration: 6.019900219s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [............X..XXX..........X...............................] duration: 6.457335117s FAIL (2025-09-08 16:18:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.457359715s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................X......................X...X.............] duration: 6.492921572s FAIL (2025-09-08 16:17:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:08:24.413421670 (50h40m0.4529948s) [upload=44521]: Starting upload Sep 8 16:08:24.413511181 (50h40m0.453084301s) [upload=44521]: Done starting upload Sep 8 16:08:24.413613276 (50h40m0.453186397s) [upload=44521]: Uploading segments... Sep 8 16:08:24.413756059 (50h40m0.45332918s) [upload=44521]: Got next segment. Inline: false Sep 8 16:08:24.413786748 (50h40m0.453359858s) [upload=44521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:08:24.413791226 (50h40m0.453364347s) [upload=44521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:08:24.688933263 (50h40m0.728506383s) [seg_pos={0 0}, upload=44521]: Begin upload segment... Sep 8 16:08:24.688998427 (50h40m0.728571598s) [seg_pos={0 0}, upload=44521]: Done begin upload segment. Sep 8 16:08:24.689039316 (50h40m0.728612436s) [noise=true , address=77.90.31.14:4662 , piece_id=XWP5AWV3IPPDI2KXSRIUO5PV733NRUGUW5GSQEXQO33DKDD2URFA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.689048103 (50h40m0.728621223s) [noise=true , address=77.90.31.15:4699 , piece_id=YK7QSHB244GAUDWCLFBGRDRTBAXKMS3PCWMKHVCDJGL7NHX5U4AA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.689552779 (50h40m0.729125899s) [noise=true , address=77.90.31.19:4843 , piece_id=KCEL6V3JZRJQ6A7NOZQ4P2KONTQAJNTCJNF4NIDBMSH33GPJJQCA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.689868502 (50h40m0.729441623s) [noise=true , address=145.79.227.3:4151 , piece_id=N5X5OBQVN5TWYUUACQ46TJQU3WAQB4OONRNIZTMUTDM42ALZNMHQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690135944 (50h40m0.729709064s) [noise=true , address=45.91.116.12:4225 , piece_id=5TK5AFJ37LD6APCYI77U7BC5ZSJFJH2S4QYKMKGAT3IPS3S7UWFQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690473119 (50h40m0.730046249s) [noise=true , address=77.90.31.10:4518 , piece_id=U34YLH6YMKT4EMX22EY2FXUMTDX4FN6SKQ2PTQT7YQEVURUSUZDA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690771941 (50h40m0.730345061s) [noise=true , address=45.91.116.13:4153 , piece_id=NTNPJURMYQWIXPG27T22CIHMB4YSYXBWYFUOZ2FGNQ5SXTIGPKZQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690987404 (50h40m0.730560524s) [noise=true , address=45.91.116.7:4011 , piece_id=YPFV6N5TS5FXAJW6VZGRNF55E5RSA5BKSVJWC7H2SJUBKPVH6QVA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691202235 (50h40m0.730775345s) [noise=true , address=45.91.116.11:4119 , piece_id=3FQTKSX5MNAIQPXHHIDQGCOHJ3WBUJXO3ICSBB2JRRDRXWYXMSVA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691411375 (50h40m0.730984495s) [noise=true , address=45.91.116.14:4191 , piece_id=WYCYDMHK4DOR2SDVBZURVSM4WDKIOWDUJG7BS5E4GAXNDOXR57XA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691598322 (50h40m0.731171442s) [noise=true , address=77.90.31.13:4627 , piece_id=GWVGPZYZJAFETOYNIOW27F3YEIUTBO6VJTPRDW4SBNW6HWJTURUA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691807092 (50h40m0.731380212s) [noise=true , address=77.90.31.21:4879 , piece_id=H23WENKOHGIJKB3PQG5X7QCDCAIA4SO4FHRW7ZOB75EYDSJVQN7Q, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692009178 (50h40m0.731582298s) [noise=true , address=45.91.116.10:4082 , piece_id=IVQ6YJQGVK7QCXW64HL4VXZF5IKKXRPJ5A22BMHULFAY4IT3LAYA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692211446 (50h40m0.731784566s) [noise=true , address=45.91.116.11:4118 , piece_id=TF5UDY4CJ6YRLH5IUEXQ7OYBTSFG4YVH4P7DD6R344VGPT7UCO3A, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692403824 (50h40m0.731976944s) [noise=true , address=45.91.116.6:4973 , piece_id=735AY7MZQRVG5Q4XMPT5GBTKGYCDX6NL34RCNIOMTIAUHNBOCWQA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692591674 (50h40m0.732164794s) [noise=true , address=45.91.116.5:4299 , piece_id=B7XEFUQTEH7PSN6XSM5IWKJHAAA23W2U4JTCFZ3C5UAFG7X42SHQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:26.488618932 (50h40m2.528192062s) [upload=44521]: Next returned nil segment Sep 8 16:08:26.488638038 (50h40m2.528211158s) [upload=44521]: Waiting for error group managing segments... Sep 8 16:08:26.782849935 (50h40m2.822423055s) [noise=true , address=45.91.116.6:4973 , piece_id=735AY7MZQRVG5Q4XMPT5GBTKGYCDX6NL34RCNIOMTIAUHNBOCWQA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:26.966056559 (50h40m3.005629679s) [noise=true , address=77.90.31.14:4662 , piece_id=XWP5AWV3IPPDI2KXSRIUO5PV733NRUGUW5GSQEXQO33DKDD2URFA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.035362168 (50h40m3.074935288s) [noise=true , address=45.91.116.11:4118 , piece_id=TF5UDY4CJ6YRLH5IUEXQ7OYBTSFG4YVH4P7DD6R344VGPT7UCO3A, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.097946837 (50h40m3.137519957s) [noise=true , address=145.79.227.3:4151 , piece_id=N5X5OBQVN5TWYUUACQ46TJQU3WAQB4OONRNIZTMUTDM42ALZNMHQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.160333006 (50h40m3.199906126s) [noise=true , address=77.90.31.10:4518 , piece_id=U34YLH6YMKT4EMX22EY2FXUMTDX4FN6SKQ2PTQT7YQEVURUSUZDA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.250934906 (50h40m3.290508026s) [noise=true , address=45.91.116.12:4225 , piece_id=5TK5AFJ37LD6APCYI77U7BC5ZSJFJH2S4QYKMKGAT3IPS3S7UWFQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.257381642 (50h40m3.296954762s) [noise=true , address=45.91.116.11:4119 , piece_id=3FQTKSX5MNAIQPXHHIDQGCOHJ3WBUJXO3ICSBB2JRRDRXWYXMSVA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.298283861 (50h40m3.337856981s) [noise=true , address=77.90.31.15:4699 , piece_id=YK7QSHB244GAUDWCLFBGRDRTBAXKMS3PCWMKHVCDJGL7NHX5U4AA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.329488312 (50h40m3.369061432s) [noise=true , address=45.91.116.5:4299 , piece_id=B7XEFUQTEH7PSN6XSM5IWKJHAAA23W2U4JTCFZ3C5UAFG7X42SHQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.338479798 (50h40m3.378052908s) [noise=true , address=45.91.116.7:4011 , piece_id=YPFV6N5TS5FXAJW6VZGRNF55E5RSA5BKSVJWC7H2SJUBKPVH6QVA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.369023604 (50h40m3.408596724s) [noise=true , address=77.90.31.21:4879 , piece_id=H23WENKOHGIJKB3PQG5X7QCDCAIA4SO4FHRW7ZOB75EYDSJVQN7Q, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:28.027636399 (50h40m4.067209529s) [noise=true , address=77.90.31.19:4843 , piece_id=KCEL6V3JZRJQ6A7NOZQ4P2KONTQAJNTCJNF4NIDBMSH33GPJJQCA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:28.088958591 (50h40m4.128531721s) [noise=true , address=45.91.116.14:4191 , piece_id=WYCYDMHK4DOR2SDVBZURVSM4WDKIOWDUJG7BS5E4GAXNDOXR57XA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:28.088984380 (50h40m4.1285575s) [seg_pos={0 0}, upload=44521]: Segment reached optimal threshold of 13 pieces. Sep 8 16:08:28.089102657 (50h40m4.128675777s) [noise=true , address=77.90.31.13:4627 , piece_id=GWVGPZYZJAFETOYNIOW27F3YEIUTBO6VJTPRDW4SBNW6HWJTURUA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB): protocol: expected piece hash; context canceled; EOF Sep 8 16:08:28.089147282 (50h40m4.128720402s) [noise=true , address=45.91.116.13:4153 , piece_id=NTNPJURMYQWIXPG27T22CIHMB4YSYXBWYFUOZ2FGNQ5SXTIGPKZQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b): protocol: expected piece hash; context canceled; EOF Sep 8 16:08:28.089173873 (50h40m4.128746993s) [noise=true , address=45.91.116.10:4082 , piece_id=IVQ6YJQGVK7QCXW64HL4VXZF5IKKXRPJ5A22BMHULFAY4IT3LAYA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Sep 8 16:08:28.089283673 (50h40m4.128856793s) [seg_pos={0 0}, upload=44521]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:08:28.089409153 (50h40m4.128982283s) [upload=44521]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:08:28.089435003 (50h40m4.129008133s) [upload=44521]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:08:28.534343825 (50h40m4.573916945s) [upload=44521]: Done uploading segments... Sep 8 16:08:28.534364004 (50h40m4.573937124s) [upload=44521]: Upload finished. err: <nil> Duration: 6.492949264s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.............X..XXX..........X..............................] duration: 6.567794919s FAIL (2025-09-08 16:17:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.567827491s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................X......................X...X............] duration: 19.710549292s FAIL (2025-09-08 16:17:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:08:24.413421670 (50h40m0.4529948s) [upload=44521]: Starting upload Sep 8 16:08:24.413511181 (50h40m0.453084301s) [upload=44521]: Done starting upload Sep 8 16:08:24.413613276 (50h40m0.453186397s) [upload=44521]: Uploading segments... Sep 8 16:08:24.413756059 (50h40m0.45332918s) [upload=44521]: Got next segment. Inline: false Sep 8 16:08:24.413786748 (50h40m0.453359858s) [upload=44521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:08:24.413791226 (50h40m0.453364347s) [upload=44521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:08:24.688933263 (50h40m0.728506383s) [seg_pos={0 0}, upload=44521]: Begin upload segment... Sep 8 16:08:24.688998427 (50h40m0.728571598s) [seg_pos={0 0}, upload=44521]: Done begin upload segment. Sep 8 16:08:24.689039316 (50h40m0.728612436s) [noise=true , address=77.90.31.14:4662 , piece_id=XWP5AWV3IPPDI2KXSRIUO5PV733NRUGUW5GSQEXQO33DKDD2URFA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.689048103 (50h40m0.728621223s) [noise=true , address=77.90.31.15:4699 , piece_id=YK7QSHB244GAUDWCLFBGRDRTBAXKMS3PCWMKHVCDJGL7NHX5U4AA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.689552779 (50h40m0.729125899s) [noise=true , address=77.90.31.19:4843 , piece_id=KCEL6V3JZRJQ6A7NOZQ4P2KONTQAJNTCJNF4NIDBMSH33GPJJQCA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.689868502 (50h40m0.729441623s) [noise=true , address=145.79.227.3:4151 , piece_id=N5X5OBQVN5TWYUUACQ46TJQU3WAQB4OONRNIZTMUTDM42ALZNMHQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690135944 (50h40m0.729709064s) [noise=true , address=45.91.116.12:4225 , piece_id=5TK5AFJ37LD6APCYI77U7BC5ZSJFJH2S4QYKMKGAT3IPS3S7UWFQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690473119 (50h40m0.730046249s) [noise=true , address=77.90.31.10:4518 , piece_id=U34YLH6YMKT4EMX22EY2FXUMTDX4FN6SKQ2PTQT7YQEVURUSUZDA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690771941 (50h40m0.730345061s) [noise=true , address=45.91.116.13:4153 , piece_id=NTNPJURMYQWIXPG27T22CIHMB4YSYXBWYFUOZ2FGNQ5SXTIGPKZQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690987404 (50h40m0.730560524s) [noise=true , address=45.91.116.7:4011 , piece_id=YPFV6N5TS5FXAJW6VZGRNF55E5RSA5BKSVJWC7H2SJUBKPVH6QVA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691202235 (50h40m0.730775345s) [noise=true , address=45.91.116.11:4119 , piece_id=3FQTKSX5MNAIQPXHHIDQGCOHJ3WBUJXO3ICSBB2JRRDRXWYXMSVA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691411375 (50h40m0.730984495s) [noise=true , address=45.91.116.14:4191 , piece_id=WYCYDMHK4DOR2SDVBZURVSM4WDKIOWDUJG7BS5E4GAXNDOXR57XA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691598322 (50h40m0.731171442s) [noise=true , address=77.90.31.13:4627 , piece_id=GWVGPZYZJAFETOYNIOW27F3YEIUTBO6VJTPRDW4SBNW6HWJTURUA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691807092 (50h40m0.731380212s) [noise=true , address=77.90.31.21:4879 , piece_id=H23WENKOHGIJKB3PQG5X7QCDCAIA4SO4FHRW7ZOB75EYDSJVQN7Q, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692009178 (50h40m0.731582298s) [noise=true , address=45.91.116.10:4082 , piece_id=IVQ6YJQGVK7QCXW64HL4VXZF5IKKXRPJ5A22BMHULFAY4IT3LAYA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692211446 (50h40m0.731784566s) [noise=true , address=45.91.116.11:4118 , piece_id=TF5UDY4CJ6YRLH5IUEXQ7OYBTSFG4YVH4P7DD6R344VGPT7UCO3A, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692403824 (50h40m0.731976944s) [noise=true , address=45.91.116.6:4973 , piece_id=735AY7MZQRVG5Q4XMPT5GBTKGYCDX6NL34RCNIOMTIAUHNBOCWQA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692591674 (50h40m0.732164794s) [noise=true , address=45.91.116.5:4299 , piece_id=B7XEFUQTEH7PSN6XSM5IWKJHAAA23W2U4JTCFZ3C5UAFG7X42SHQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:26.488618932 (50h40m2.528192062s) [upload=44521]: Next returned nil segment Sep 8 16:08:26.488638038 (50h40m2.528211158s) [upload=44521]: Waiting for error group managing segments... Sep 8 16:08:26.782849935 (50h40m2.822423055s) [noise=true , address=45.91.116.6:4973 , piece_id=735AY7MZQRVG5Q4XMPT5GBTKGYCDX6NL34RCNIOMTIAUHNBOCWQA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:26.966056559 (50h40m3.005629679s) [noise=true , address=77.90.31.14:4662 , piece_id=XWP5AWV3IPPDI2KXSRIUO5PV733NRUGUW5GSQEXQO33DKDD2URFA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.035362168 (50h40m3.074935288s) [noise=true , address=45.91.116.11:4118 , piece_id=TF5UDY4CJ6YRLH5IUEXQ7OYBTSFG4YVH4P7DD6R344VGPT7UCO3A, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.097946837 (50h40m3.137519957s) [noise=true , address=145.79.227.3:4151 , piece_id=N5X5OBQVN5TWYUUACQ46TJQU3WAQB4OONRNIZTMUTDM42ALZNMHQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.160333006 (50h40m3.199906126s) [noise=true , address=77.90.31.10:4518 , piece_id=U34YLH6YMKT4EMX22EY2FXUMTDX4FN6SKQ2PTQT7YQEVURUSUZDA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.250934906 (50h40m3.290508026s) [noise=true , address=45.91.116.12:4225 , piece_id=5TK5AFJ37LD6APCYI77U7BC5ZSJFJH2S4QYKMKGAT3IPS3S7UWFQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.257381642 (50h40m3.296954762s) [noise=true , address=45.91.116.11:4119 , piece_id=3FQTKSX5MNAIQPXHHIDQGCOHJ3WBUJXO3ICSBB2JRRDRXWYXMSVA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.298283861 (50h40m3.337856981s) [noise=true , address=77.90.31.15:4699 , piece_id=YK7QSHB244GAUDWCLFBGRDRTBAXKMS3PCWMKHVCDJGL7NHX5U4AA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.329488312 (50h40m3.369061432s) [noise=true , address=45.91.116.5:4299 , piece_id=B7XEFUQTEH7PSN6XSM5IWKJHAAA23W2U4JTCFZ3C5UAFG7X42SHQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.338479798 (50h40m3.378052908s) [noise=true , address=45.91.116.7:4011 , piece_id=YPFV6N5TS5FXAJW6VZGRNF55E5RSA5BKSVJWC7H2SJUBKPVH6QVA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.369023604 (50h40m3.408596724s) [noise=true , address=77.90.31.21:4879 , piece_id=H23WENKOHGIJKB3PQG5X7QCDCAIA4SO4FHRW7ZOB75EYDSJVQN7Q, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:28.027636399 (50h40m4.067209529s) [noise=true , address=77.90.31.19:4843 , piece_id=KCEL6V3JZRJQ6A7NOZQ4P2KONTQAJNTCJNF4NIDBMSH33GPJJQCA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:28.088958591 (50h40m4.128531721s) [noise=true , address=45.91.116.14:4191 , piece_id=WYCYDMHK4DOR2SDVBZURVSM4WDKIOWDUJG7BS5E4GAXNDOXR57XA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:28.088984380 (50h40m4.1285575s) [seg_pos={0 0}, upload=44521]: Segment reached optimal threshold of 13 pieces. Sep 8 16:08:28.089102657 (50h40m4.128675777s) [noise=true , address=77.90.31.13:4627 , piece_id=GWVGPZYZJAFETOYNIOW27F3YEIUTBO6VJTPRDW4SBNW6HWJTURUA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB): protocol: expected piece hash; context canceled; EOF Sep 8 16:08:28.089147282 (50h40m4.128720402s) [noise=true , address=45.91.116.13:4153 , piece_id=NTNPJURMYQWIXPG27T22CIHMB4YSYXBWYFUOZ2FGNQ5SXTIGPKZQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b): protocol: expected piece hash; context canceled; EOF Sep 8 16:08:28.089173873 (50h40m4.128746993s) [noise=true , address=45.91.116.10:4082 , piece_id=IVQ6YJQGVK7QCXW64HL4VXZF5IKKXRPJ5A22BMHULFAY4IT3LAYA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Sep 8 16:08:28.089283673 (50h40m4.128856793s) [seg_pos={0 0}, upload=44521]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:08:28.089409153 (50h40m4.128982283s) [upload=44521]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:08:28.089435003 (50h40m4.129008133s) [upload=44521]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:08:28.534343825 (50h40m4.573916945s) [upload=44521]: Done uploading segments... Sep 8 16:08:28.534364004 (50h40m4.573937124s) [upload=44521]: Upload finished. err: <nil> Duration: 19.710585693s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................X......................X...X...........] duration: 7.504265057s FAIL (2025-09-08 16:16:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:08:24.413421670 (50h40m0.4529948s) [upload=44521]: Starting upload Sep 8 16:08:24.413511181 (50h40m0.453084301s) [upload=44521]: Done starting upload Sep 8 16:08:24.413613276 (50h40m0.453186397s) [upload=44521]: Uploading segments... Sep 8 16:08:24.413756059 (50h40m0.45332918s) [upload=44521]: Got next segment. Inline: false Sep 8 16:08:24.413786748 (50h40m0.453359858s) [upload=44521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:08:24.413791226 (50h40m0.453364347s) [upload=44521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:08:24.688933263 (50h40m0.728506383s) [seg_pos={0 0}, upload=44521]: Begin upload segment... Sep 8 16:08:24.688998427 (50h40m0.728571598s) [seg_pos={0 0}, upload=44521]: Done begin upload segment. Sep 8 16:08:24.689039316 (50h40m0.728612436s) [noise=true , address=77.90.31.14:4662 , piece_id=XWP5AWV3IPPDI2KXSRIUO5PV733NRUGUW5GSQEXQO33DKDD2URFA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.689048103 (50h40m0.728621223s) [noise=true , address=77.90.31.15:4699 , piece_id=YK7QSHB244GAUDWCLFBGRDRTBAXKMS3PCWMKHVCDJGL7NHX5U4AA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.689552779 (50h40m0.729125899s) [noise=true , address=77.90.31.19:4843 , piece_id=KCEL6V3JZRJQ6A7NOZQ4P2KONTQAJNTCJNF4NIDBMSH33GPJJQCA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.689868502 (50h40m0.729441623s) [noise=true , address=145.79.227.3:4151 , piece_id=N5X5OBQVN5TWYUUACQ46TJQU3WAQB4OONRNIZTMUTDM42ALZNMHQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690135944 (50h40m0.729709064s) [noise=true , address=45.91.116.12:4225 , piece_id=5TK5AFJ37LD6APCYI77U7BC5ZSJFJH2S4QYKMKGAT3IPS3S7UWFQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690473119 (50h40m0.730046249s) [noise=true , address=77.90.31.10:4518 , piece_id=U34YLH6YMKT4EMX22EY2FXUMTDX4FN6SKQ2PTQT7YQEVURUSUZDA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690771941 (50h40m0.730345061s) [noise=true , address=45.91.116.13:4153 , piece_id=NTNPJURMYQWIXPG27T22CIHMB4YSYXBWYFUOZ2FGNQ5SXTIGPKZQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690987404 (50h40m0.730560524s) [noise=true , address=45.91.116.7:4011 , piece_id=YPFV6N5TS5FXAJW6VZGRNF55E5RSA5BKSVJWC7H2SJUBKPVH6QVA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691202235 (50h40m0.730775345s) [noise=true , address=45.91.116.11:4119 , piece_id=3FQTKSX5MNAIQPXHHIDQGCOHJ3WBUJXO3ICSBB2JRRDRXWYXMSVA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691411375 (50h40m0.730984495s) [noise=true , address=45.91.116.14:4191 , piece_id=WYCYDMHK4DOR2SDVBZURVSM4WDKIOWDUJG7BS5E4GAXNDOXR57XA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691598322 (50h40m0.731171442s) [noise=true , address=77.90.31.13:4627 , piece_id=GWVGPZYZJAFETOYNIOW27F3YEIUTBO6VJTPRDW4SBNW6HWJTURUA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691807092 (50h40m0.731380212s) [noise=true , address=77.90.31.21:4879 , piece_id=H23WENKOHGIJKB3PQG5X7QCDCAIA4SO4FHRW7ZOB75EYDSJVQN7Q, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692009178 (50h40m0.731582298s) [noise=true , address=45.91.116.10:4082 , piece_id=IVQ6YJQGVK7QCXW64HL4VXZF5IKKXRPJ5A22BMHULFAY4IT3LAYA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692211446 (50h40m0.731784566s) [noise=true , address=45.91.116.11:4118 , piece_id=TF5UDY4CJ6YRLH5IUEXQ7OYBTSFG4YVH4P7DD6R344VGPT7UCO3A, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692403824 (50h40m0.731976944s) [noise=true , address=45.91.116.6:4973 , piece_id=735AY7MZQRVG5Q4XMPT5GBTKGYCDX6NL34RCNIOMTIAUHNBOCWQA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692591674 (50h40m0.732164794s) [noise=true , address=45.91.116.5:4299 , piece_id=B7XEFUQTEH7PSN6XSM5IWKJHAAA23W2U4JTCFZ3C5UAFG7X42SHQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:26.488618932 (50h40m2.528192062s) [upload=44521]: Next returned nil segment Sep 8 16:08:26.488638038 (50h40m2.528211158s) [upload=44521]: Waiting for error group managing segments... Sep 8 16:08:26.782849935 (50h40m2.822423055s) [noise=true , address=45.91.116.6:4973 , piece_id=735AY7MZQRVG5Q4XMPT5GBTKGYCDX6NL34RCNIOMTIAUHNBOCWQA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:26.966056559 (50h40m3.005629679s) [noise=true , address=77.90.31.14:4662 , piece_id=XWP5AWV3IPPDI2KXSRIUO5PV733NRUGUW5GSQEXQO33DKDD2URFA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.035362168 (50h40m3.074935288s) [noise=true , address=45.91.116.11:4118 , piece_id=TF5UDY4CJ6YRLH5IUEXQ7OYBTSFG4YVH4P7DD6R344VGPT7UCO3A, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.097946837 (50h40m3.137519957s) [noise=true , address=145.79.227.3:4151 , piece_id=N5X5OBQVN5TWYUUACQ46TJQU3WAQB4OONRNIZTMUTDM42ALZNMHQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.160333006 (50h40m3.199906126s) [noise=true , address=77.90.31.10:4518 , piece_id=U34YLH6YMKT4EMX22EY2FXUMTDX4FN6SKQ2PTQT7YQEVURUSUZDA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.250934906 (50h40m3.290508026s) [noise=true , address=45.91.116.12:4225 , piece_id=5TK5AFJ37LD6APCYI77U7BC5ZSJFJH2S4QYKMKGAT3IPS3S7UWFQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.257381642 (50h40m3.296954762s) [noise=true , address=45.91.116.11:4119 , piece_id=3FQTKSX5MNAIQPXHHIDQGCOHJ3WBUJXO3ICSBB2JRRDRXWYXMSVA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.298283861 (50h40m3.337856981s) [noise=true , address=77.90.31.15:4699 , piece_id=YK7QSHB244GAUDWCLFBGRDRTBAXKMS3PCWMKHVCDJGL7NHX5U4AA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.329488312 (50h40m3.369061432s) [noise=true , address=45.91.116.5:4299 , piece_id=B7XEFUQTEH7PSN6XSM5IWKJHAAA23W2U4JTCFZ3C5UAFG7X42SHQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.338479798 (50h40m3.378052908s) [noise=true , address=45.91.116.7:4011 , piece_id=YPFV6N5TS5FXAJW6VZGRNF55E5RSA5BKSVJWC7H2SJUBKPVH6QVA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.369023604 (50h40m3.408596724s) [noise=true , address=77.90.31.21:4879 , piece_id=H23WENKOHGIJKB3PQG5X7QCDCAIA4SO4FHRW7ZOB75EYDSJVQN7Q, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:28.027636399 (50h40m4.067209529s) [noise=true , address=77.90.31.19:4843 , piece_id=KCEL6V3JZRJQ6A7NOZQ4P2KONTQAJNTCJNF4NIDBMSH33GPJJQCA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:28.088958591 (50h40m4.128531721s) [noise=true , address=45.91.116.14:4191 , piece_id=WYCYDMHK4DOR2SDVBZURVSM4WDKIOWDUJG7BS5E4GAXNDOXR57XA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:28.088984380 (50h40m4.1285575s) [seg_pos={0 0}, upload=44521]: Segment reached optimal threshold of 13 pieces. Sep 8 16:08:28.089102657 (50h40m4.128675777s) [noise=true , address=77.90.31.13:4627 , piece_id=GWVGPZYZJAFETOYNIOW27F3YEIUTBO6VJTPRDW4SBNW6HWJTURUA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB): protocol: expected piece hash; context canceled; EOF Sep 8 16:08:28.089147282 (50h40m4.128720402s) [noise=true , address=45.91.116.13:4153 , piece_id=NTNPJURMYQWIXPG27T22CIHMB4YSYXBWYFUOZ2FGNQ5SXTIGPKZQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b): protocol: expected piece hash; context canceled; EOF Sep 8 16:08:28.089173873 (50h40m4.128746993s) [noise=true , address=45.91.116.10:4082 , piece_id=IVQ6YJQGVK7QCXW64HL4VXZF5IKKXRPJ5A22BMHULFAY4IT3LAYA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Sep 8 16:08:28.089283673 (50h40m4.128856793s) [seg_pos={0 0}, upload=44521]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:08:28.089409153 (50h40m4.128982283s) [upload=44521]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:08:28.089435003 (50h40m4.129008133s) [upload=44521]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:08:28.534343825 (50h40m4.573916945s) [upload=44521]: Done uploading segments... Sep 8 16:08:28.534364004 (50h40m4.573937124s) [upload=44521]: Upload finished. err: <nil> Duration: 7.504297538s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..............X..XXX..........X.............................] duration: 7.604864897s FAIL (2025-09-08 16:16:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.604900465s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...............X..XXX..........X............................] duration: 6.874468545s FAIL (2025-09-08 16:16:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.874491129s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [X.....................X......................X...X..........] duration: 7.217083142s FAIL (2025-09-08 16:16:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 8 16:08:24.413421670 (50h40m0.4529948s) [upload=44521]: Starting upload Sep 8 16:08:24.413511181 (50h40m0.453084301s) [upload=44521]: Done starting upload Sep 8 16:08:24.413613276 (50h40m0.453186397s) [upload=44521]: Uploading segments... Sep 8 16:08:24.413756059 (50h40m0.45332918s) [upload=44521]: Got next segment. Inline: false Sep 8 16:08:24.413786748 (50h40m0.453359858s) [upload=44521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 8 16:08:24.413791226 (50h40m0.453364347s) [upload=44521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 8 16:08:24.688933263 (50h40m0.728506383s) [seg_pos={0 0}, upload=44521]: Begin upload segment... Sep 8 16:08:24.688998427 (50h40m0.728571598s) [seg_pos={0 0}, upload=44521]: Done begin upload segment. Sep 8 16:08:24.689039316 (50h40m0.728612436s) [noise=true , address=77.90.31.14:4662 , piece_id=XWP5AWV3IPPDI2KXSRIUO5PV733NRUGUW5GSQEXQO33DKDD2URFA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.689048103 (50h40m0.728621223s) [noise=true , address=77.90.31.15:4699 , piece_id=YK7QSHB244GAUDWCLFBGRDRTBAXKMS3PCWMKHVCDJGL7NHX5U4AA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.689552779 (50h40m0.729125899s) [noise=true , address=77.90.31.19:4843 , piece_id=KCEL6V3JZRJQ6A7NOZQ4P2KONTQAJNTCJNF4NIDBMSH33GPJJQCA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.689868502 (50h40m0.729441623s) [noise=true , address=145.79.227.3:4151 , piece_id=N5X5OBQVN5TWYUUACQ46TJQU3WAQB4OONRNIZTMUTDM42ALZNMHQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690135944 (50h40m0.729709064s) [noise=true , address=45.91.116.12:4225 , piece_id=5TK5AFJ37LD6APCYI77U7BC5ZSJFJH2S4QYKMKGAT3IPS3S7UWFQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690473119 (50h40m0.730046249s) [noise=true , address=77.90.31.10:4518 , piece_id=U34YLH6YMKT4EMX22EY2FXUMTDX4FN6SKQ2PTQT7YQEVURUSUZDA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690771941 (50h40m0.730345061s) [noise=true , address=45.91.116.13:4153 , piece_id=NTNPJURMYQWIXPG27T22CIHMB4YSYXBWYFUOZ2FGNQ5SXTIGPKZQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.690987404 (50h40m0.730560524s) [noise=true , address=45.91.116.7:4011 , piece_id=YPFV6N5TS5FXAJW6VZGRNF55E5RSA5BKSVJWC7H2SJUBKPVH6QVA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691202235 (50h40m0.730775345s) [noise=true , address=45.91.116.11:4119 , piece_id=3FQTKSX5MNAIQPXHHIDQGCOHJ3WBUJXO3ICSBB2JRRDRXWYXMSVA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691411375 (50h40m0.730984495s) [noise=true , address=45.91.116.14:4191 , piece_id=WYCYDMHK4DOR2SDVBZURVSM4WDKIOWDUJG7BS5E4GAXNDOXR57XA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691598322 (50h40m0.731171442s) [noise=true , address=77.90.31.13:4627 , piece_id=GWVGPZYZJAFETOYNIOW27F3YEIUTBO6VJTPRDW4SBNW6HWJTURUA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.691807092 (50h40m0.731380212s) [noise=true , address=77.90.31.21:4879 , piece_id=H23WENKOHGIJKB3PQG5X7QCDCAIA4SO4FHRW7ZOB75EYDSJVQN7Q, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692009178 (50h40m0.731582298s) [noise=true , address=45.91.116.10:4082 , piece_id=IVQ6YJQGVK7QCXW64HL4VXZF5IKKXRPJ5A22BMHULFAY4IT3LAYA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692211446 (50h40m0.731784566s) [noise=true , address=45.91.116.11:4118 , piece_id=TF5UDY4CJ6YRLH5IUEXQ7OYBTSFG4YVH4P7DD6R344VGPT7UCO3A, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692403824 (50h40m0.731976944s) [noise=true , address=45.91.116.6:4973 , piece_id=735AY7MZQRVG5Q4XMPT5GBTKGYCDX6NL34RCNIOMTIAUHNBOCWQA, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:24.692591674 (50h40m0.732164794s) [noise=true , address=45.91.116.5:4299 , piece_id=B7XEFUQTEH7PSN6XSM5IWKJHAAA23W2U4JTCFZ3C5UAFG7X42SHQ, seg_pos={0 0}, upload=44521]: Uploading piece... Sep 8 16:08:26.488618932 (50h40m2.528192062s) [upload=44521]: Next returned nil segment Sep 8 16:08:26.488638038 (50h40m2.528211158s) [upload=44521]: Waiting for error group managing segments... Sep 8 16:08:26.782849935 (50h40m2.822423055s) [noise=true , address=45.91.116.6:4973 , piece_id=735AY7MZQRVG5Q4XMPT5GBTKGYCDX6NL34RCNIOMTIAUHNBOCWQA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:26.966056559 (50h40m3.005629679s) [noise=true , address=77.90.31.14:4662 , piece_id=XWP5AWV3IPPDI2KXSRIUO5PV733NRUGUW5GSQEXQO33DKDD2URFA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.035362168 (50h40m3.074935288s) [noise=true , address=45.91.116.11:4118 , piece_id=TF5UDY4CJ6YRLH5IUEXQ7OYBTSFG4YVH4P7DD6R344VGPT7UCO3A, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.097946837 (50h40m3.137519957s) [noise=true , address=145.79.227.3:4151 , piece_id=N5X5OBQVN5TWYUUACQ46TJQU3WAQB4OONRNIZTMUTDM42ALZNMHQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.160333006 (50h40m3.199906126s) [noise=true , address=77.90.31.10:4518 , piece_id=U34YLH6YMKT4EMX22EY2FXUMTDX4FN6SKQ2PTQT7YQEVURUSUZDA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.250934906 (50h40m3.290508026s) [noise=true , address=45.91.116.12:4225 , piece_id=5TK5AFJ37LD6APCYI77U7BC5ZSJFJH2S4QYKMKGAT3IPS3S7UWFQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.257381642 (50h40m3.296954762s) [noise=true , address=45.91.116.11:4119 , piece_id=3FQTKSX5MNAIQPXHHIDQGCOHJ3WBUJXO3ICSBB2JRRDRXWYXMSVA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.298283861 (50h40m3.337856981s) [noise=true , address=77.90.31.15:4699 , piece_id=YK7QSHB244GAUDWCLFBGRDRTBAXKMS3PCWMKHVCDJGL7NHX5U4AA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.329488312 (50h40m3.369061432s) [noise=true , address=45.91.116.5:4299 , piece_id=B7XEFUQTEH7PSN6XSM5IWKJHAAA23W2U4JTCFZ3C5UAFG7X42SHQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.338479798 (50h40m3.378052908s) [noise=true , address=45.91.116.7:4011 , piece_id=YPFV6N5TS5FXAJW6VZGRNF55E5RSA5BKSVJWC7H2SJUBKPVH6QVA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:27.369023604 (50h40m3.408596724s) [noise=true , address=77.90.31.21:4879 , piece_id=H23WENKOHGIJKB3PQG5X7QCDCAIA4SO4FHRW7ZOB75EYDSJVQN7Q, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:28.027636399 (50h40m4.067209529s) [noise=true , address=77.90.31.19:4843 , piece_id=KCEL6V3JZRJQ6A7NOZQ4P2KONTQAJNTCJNF4NIDBMSH33GPJJQCA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:28.088958591 (50h40m4.128531721s) [noise=true , address=45.91.116.14:4191 , piece_id=WYCYDMHK4DOR2SDVBZURVSM4WDKIOWDUJG7BS5E4GAXNDOXR57XA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: <nil> Sep 8 16:08:28.088984380 (50h40m4.1285575s) [seg_pos={0 0}, upload=44521]: Segment reached optimal threshold of 13 pieces. Sep 8 16:08:28.089102657 (50h40m4.128675777s) [noise=true , address=77.90.31.13:4627 , piece_id=GWVGPZYZJAFETOYNIOW27F3YEIUTBO6VJTPRDW4SBNW6HWJTURUA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB): protocol: expected piece hash; context canceled; EOF Sep 8 16:08:28.089147282 (50h40m4.128720402s) [noise=true , address=45.91.116.13:4153 , piece_id=NTNPJURMYQWIXPG27T22CIHMB4YSYXBWYFUOZ2FGNQ5SXTIGPKZQ, seg_pos={0 0}, upload=44521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b): protocol: expected piece hash; context canceled; EOF Sep 8 16:08:28.089173873 (50h40m4.128746993s) [noise=true , address=45.91.116.10:4082 , piece_id=IVQ6YJQGVK7QCXW64HL4VXZF5IKKXRPJ5A22BMHULFAY4IT3LAYA, seg_pos={0 0}, upload=44521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Sep 8 16:08:28.089283673 (50h40m4.128856793s) [seg_pos={0 0}, upload=44521]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 8 16:08:28.089409153 (50h40m4.128982283s) [upload=44521]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 8 16:08:28.089435003 (50h40m4.129008133s) [upload=44521]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 8 16:08:28.534343825 (50h40m4.573916945s) [upload=44521]: Done uploading segments... Sep 8 16:08:28.534364004 (50h40m4.573937124s) [upload=44521]: Upload finished. err: <nil> Duration: 7.217116215s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [................X..XXX..........X...........................] duration: 8.05896886s FAIL (2025-09-08 16:15:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.058998256s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)