Orbiter History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 2 out of 60 is failed (threshold=3) [....................................X....X..................] duration: 1.533516153s FAIL (2025-09-07 04:49:25 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.533543396s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............................X.....XXX.....................] duration: 5.36220591s FAIL (2025-09-07 04:48:54 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 5.3622313s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............................X.....XXX....................] duration: 5.060940066s FAIL (2025-09-07 04:48:36 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 5.060962219s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 6.205018663s FAIL (2025-09-07 04:48:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 04:23:05.779258791 (14h54m41.818831921s) [upload=13122]: Starting upload Sep 7 04:23:05.779322874 (14h54m41.818896005s) [upload=13122]: Done starting upload Sep 7 04:23:05.779427754 (14h54m41.819000874s) [upload=13122]: Uploading segments... Sep 7 04:23:05.779517276 (14h54m41.819090396s) [upload=13122]: Got next segment. Inline: false Sep 7 04:23:05.779550860 (14h54m41.81912398s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 04:23:05.779558204 (14h54m41.819131324s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 04:23:08.191697796 (14h54m44.231270916s) [seg_pos={0 0}, upload=13122]: Begin upload segment... Sep 7 04:23:08.191745878 (14h54m44.231318998s) [seg_pos={0 0}, upload=13122]: Done begin upload segment. Sep 7 04:23:08.191768891 (14h54m44.231342011s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.191773190 (14h54m44.23134631s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192136916 (14h54m44.231710026s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192145131 (14h54m44.231718251s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192406221 (14h54m44.231979331s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192664154 (14h54m44.232237274s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192901128 (14h54m44.232474248s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193209558 (14h54m44.232782678s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193449928 (14h54m44.233023048s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193686511 (14h54m44.233259621s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193897635 (14h54m44.233470755s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194095294 (14h54m44.233668414s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194288964 (14h54m44.233862084s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194473567 (14h54m44.234046687s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194726201 (14h54m44.234299321s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.196014095 (14h54m44.235587215s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:12.514659180 (14h54m48.5542323s) [upload=13122]: Next returned nil segment Sep 7 04:23:12.514677153 (14h54m48.554250273s) [upload=13122]: Waiting for error group managing segments... Sep 7 04:23:14.018528872 (14h54m50.058101992s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.233524174 (14h54m50.273097294s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.455484504 (14h54m50.495057624s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.483549776 (14h54m50.523122896s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.938110455 (14h54m50.977683575s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.992469711 (14h54m51.032042831s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.108041127 (14h54m51.147614247s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.182620684 (14h54m51.222193824s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.392802967 (14h54m51.432376087s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.425736273 (14h54m51.465309393s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.458705710 (14h54m51.49827883s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.738518791 (14h54m51.778091911s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973957025 (14h54m53.013530145s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973983115 (14h54m53.013556235s) [seg_pos={0 0}, upload=13122]: Segment reached optimal threshold of 13 pieces. Sep 7 04:23:16.974131328 (14h54m53.013704438s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Sep 7 04:23:16.974191383 (14h54m53.013764503s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection Sep 7 04:23:16.974224767 (14h54m53.013797887s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection Sep 7 04:23:16.974242441 (14h54m53.013815561s) [seg_pos={0 0}, upload=13122]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 04:23:16.974334678 (14h54m53.013907828s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 04:23:16.974341410 (14h54m53.01391454s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 04:23:17.336870127 (14h54m53.376443257s) [upload=13122]: Done uploading segments... Sep 7 04:23:17.336909021 (14h54m53.376482141s) [upload=13122]: Upload finished. err: <nil> Duration: 6.205047467s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X....X.................] duration: 1.529630949s FAIL (2025-09-07 04:48:11 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.529646649s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.060641929s FAIL (2025-09-07 04:47:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 04:23:05.779258791 (14h54m41.818831921s) [upload=13122]: Starting upload Sep 7 04:23:05.779322874 (14h54m41.818896005s) [upload=13122]: Done starting upload Sep 7 04:23:05.779427754 (14h54m41.819000874s) [upload=13122]: Uploading segments... Sep 7 04:23:05.779517276 (14h54m41.819090396s) [upload=13122]: Got next segment. Inline: false Sep 7 04:23:05.779550860 (14h54m41.81912398s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 04:23:05.779558204 (14h54m41.819131324s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 04:23:08.191697796 (14h54m44.231270916s) [seg_pos={0 0}, upload=13122]: Begin upload segment... Sep 7 04:23:08.191745878 (14h54m44.231318998s) [seg_pos={0 0}, upload=13122]: Done begin upload segment. Sep 7 04:23:08.191768891 (14h54m44.231342011s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.191773190 (14h54m44.23134631s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192136916 (14h54m44.231710026s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192145131 (14h54m44.231718251s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192406221 (14h54m44.231979331s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192664154 (14h54m44.232237274s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192901128 (14h54m44.232474248s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193209558 (14h54m44.232782678s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193449928 (14h54m44.233023048s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193686511 (14h54m44.233259621s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193897635 (14h54m44.233470755s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194095294 (14h54m44.233668414s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194288964 (14h54m44.233862084s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194473567 (14h54m44.234046687s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194726201 (14h54m44.234299321s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.196014095 (14h54m44.235587215s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:12.514659180 (14h54m48.5542323s) [upload=13122]: Next returned nil segment Sep 7 04:23:12.514677153 (14h54m48.554250273s) [upload=13122]: Waiting for error group managing segments... Sep 7 04:23:14.018528872 (14h54m50.058101992s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.233524174 (14h54m50.273097294s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.455484504 (14h54m50.495057624s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.483549776 (14h54m50.523122896s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.938110455 (14h54m50.977683575s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.992469711 (14h54m51.032042831s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.108041127 (14h54m51.147614247s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.182620684 (14h54m51.222193824s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.392802967 (14h54m51.432376087s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.425736273 (14h54m51.465309393s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.458705710 (14h54m51.49827883s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.738518791 (14h54m51.778091911s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973957025 (14h54m53.013530145s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973983115 (14h54m53.013556235s) [seg_pos={0 0}, upload=13122]: Segment reached optimal threshold of 13 pieces. Sep 7 04:23:16.974131328 (14h54m53.013704438s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Sep 7 04:23:16.974191383 (14h54m53.013764503s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection Sep 7 04:23:16.974224767 (14h54m53.013797887s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection Sep 7 04:23:16.974242441 (14h54m53.013815561s) [seg_pos={0 0}, upload=13122]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 04:23:16.974334678 (14h54m53.013907828s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 04:23:16.974341410 (14h54m53.01391454s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 04:23:17.336870127 (14h54m53.376443257s) [upload=13122]: Done uploading segments... Sep 7 04:23:17.336909021 (14h54m53.376482141s) [upload=13122]: Upload finished. err: <nil> Duration: 6.06067443s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 4 out of 60 is failed (threshold=3) [................................X.....XXX...................] duration: 5.319494964s FAIL (2025-09-07 04:47:45 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 5.319523598s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 2 out of 60 is failed (threshold=3) [......................................X....X................] duration: 1.414119337s FAIL (2025-09-07 04:47:36 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.414145026s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................................X.....XXX..................] duration: 5.349752573s FAIL (2025-09-07 04:47:27 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 5.349786076s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.244638065s FAIL (2025-09-07 04:47:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 04:23:05.779258791 (14h54m41.818831921s) [upload=13122]: Starting upload Sep 7 04:23:05.779322874 (14h54m41.818896005s) [upload=13122]: Done starting upload Sep 7 04:23:05.779427754 (14h54m41.819000874s) [upload=13122]: Uploading segments... Sep 7 04:23:05.779517276 (14h54m41.819090396s) [upload=13122]: Got next segment. Inline: false Sep 7 04:23:05.779550860 (14h54m41.81912398s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 04:23:05.779558204 (14h54m41.819131324s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 04:23:08.191697796 (14h54m44.231270916s) [seg_pos={0 0}, upload=13122]: Begin upload segment... Sep 7 04:23:08.191745878 (14h54m44.231318998s) [seg_pos={0 0}, upload=13122]: Done begin upload segment. Sep 7 04:23:08.191768891 (14h54m44.231342011s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.191773190 (14h54m44.23134631s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192136916 (14h54m44.231710026s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192145131 (14h54m44.231718251s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192406221 (14h54m44.231979331s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192664154 (14h54m44.232237274s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192901128 (14h54m44.232474248s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193209558 (14h54m44.232782678s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193449928 (14h54m44.233023048s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193686511 (14h54m44.233259621s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193897635 (14h54m44.233470755s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194095294 (14h54m44.233668414s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194288964 (14h54m44.233862084s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194473567 (14h54m44.234046687s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194726201 (14h54m44.234299321s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.196014095 (14h54m44.235587215s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:12.514659180 (14h54m48.5542323s) [upload=13122]: Next returned nil segment Sep 7 04:23:12.514677153 (14h54m48.554250273s) [upload=13122]: Waiting for error group managing segments... Sep 7 04:23:14.018528872 (14h54m50.058101992s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.233524174 (14h54m50.273097294s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.455484504 (14h54m50.495057624s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.483549776 (14h54m50.523122896s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.938110455 (14h54m50.977683575s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.992469711 (14h54m51.032042831s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.108041127 (14h54m51.147614247s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.182620684 (14h54m51.222193824s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.392802967 (14h54m51.432376087s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.425736273 (14h54m51.465309393s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.458705710 (14h54m51.49827883s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.738518791 (14h54m51.778091911s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973957025 (14h54m53.013530145s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973983115 (14h54m53.013556235s) [seg_pos={0 0}, upload=13122]: Segment reached optimal threshold of 13 pieces. Sep 7 04:23:16.974131328 (14h54m53.013704438s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Sep 7 04:23:16.974191383 (14h54m53.013764503s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection Sep 7 04:23:16.974224767 (14h54m53.013797887s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection Sep 7 04:23:16.974242441 (14h54m53.013815561s) [seg_pos={0 0}, upload=13122]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 04:23:16.974334678 (14h54m53.013907828s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 04:23:16.974341410 (14h54m53.01391454s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 04:23:17.336870127 (14h54m53.376443257s) [upload=13122]: Done uploading segments... Sep 7 04:23:17.336909021 (14h54m53.376482141s) [upload=13122]: Upload finished. err: <nil> Duration: 6.2446667s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.277210363s FAIL (2025-09-07 04:46:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 04:23:05.779258791 (14h54m41.818831921s) [upload=13122]: Starting upload Sep 7 04:23:05.779322874 (14h54m41.818896005s) [upload=13122]: Done starting upload Sep 7 04:23:05.779427754 (14h54m41.819000874s) [upload=13122]: Uploading segments... Sep 7 04:23:05.779517276 (14h54m41.819090396s) [upload=13122]: Got next segment. Inline: false Sep 7 04:23:05.779550860 (14h54m41.81912398s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 04:23:05.779558204 (14h54m41.819131324s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 04:23:08.191697796 (14h54m44.231270916s) [seg_pos={0 0}, upload=13122]: Begin upload segment... Sep 7 04:23:08.191745878 (14h54m44.231318998s) [seg_pos={0 0}, upload=13122]: Done begin upload segment. Sep 7 04:23:08.191768891 (14h54m44.231342011s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.191773190 (14h54m44.23134631s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192136916 (14h54m44.231710026s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192145131 (14h54m44.231718251s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192406221 (14h54m44.231979331s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192664154 (14h54m44.232237274s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192901128 (14h54m44.232474248s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193209558 (14h54m44.232782678s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193449928 (14h54m44.233023048s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193686511 (14h54m44.233259621s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193897635 (14h54m44.233470755s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194095294 (14h54m44.233668414s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194288964 (14h54m44.233862084s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194473567 (14h54m44.234046687s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194726201 (14h54m44.234299321s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.196014095 (14h54m44.235587215s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:12.514659180 (14h54m48.5542323s) [upload=13122]: Next returned nil segment Sep 7 04:23:12.514677153 (14h54m48.554250273s) [upload=13122]: Waiting for error group managing segments... Sep 7 04:23:14.018528872 (14h54m50.058101992s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.233524174 (14h54m50.273097294s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.455484504 (14h54m50.495057624s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.483549776 (14h54m50.523122896s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.938110455 (14h54m50.977683575s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.992469711 (14h54m51.032042831s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.108041127 (14h54m51.147614247s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.182620684 (14h54m51.222193824s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.392802967 (14h54m51.432376087s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.425736273 (14h54m51.465309393s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.458705710 (14h54m51.49827883s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.738518791 (14h54m51.778091911s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973957025 (14h54m53.013530145s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973983115 (14h54m53.013556235s) [seg_pos={0 0}, upload=13122]: Segment reached optimal threshold of 13 pieces. Sep 7 04:23:16.974131328 (14h54m53.013704438s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Sep 7 04:23:16.974191383 (14h54m53.013764503s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection Sep 7 04:23:16.974224767 (14h54m53.013797887s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection Sep 7 04:23:16.974242441 (14h54m53.013815561s) [seg_pos={0 0}, upload=13122]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 04:23:16.974334678 (14h54m53.013907828s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 04:23:16.974341410 (14h54m53.01391454s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 04:23:17.336870127 (14h54m53.376443257s) [upload=13122]: Done uploading segments... Sep 7 04:23:17.336909021 (14h54m53.376482141s) [upload=13122]: Upload finished. err: <nil> Duration: 6.277243826s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................................X.....XXX.................] duration: 4.924076822s FAIL (2025-09-07 04:46:28 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 4.924094226s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................................X.....XXX................] duration: 25.505682363s FAIL (2025-09-07 04:46:23 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 25.505708594s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X....X...............] duration: 1.430620121s FAIL (2025-09-07 04:46:22 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.430656471s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 2 out of 60 is failed (threshold=3) [........................................X....X..............] duration: 1.438541509s FAIL (2025-09-07 04:46:00 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.438566318s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........................................X....X.............] duration: 1.420777807s FAIL (2025-09-07 04:45:28 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.420825479s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................................X.....XXX...............] duration: 5.249177207s FAIL (2025-09-07 04:45:26 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 5.249205271s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.07055623s FAIL (2025-09-07 04:45:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 04:23:05.779258791 (14h54m41.818831921s) [upload=13122]: Starting upload Sep 7 04:23:05.779322874 (14h54m41.818896005s) [upload=13122]: Done starting upload Sep 7 04:23:05.779427754 (14h54m41.819000874s) [upload=13122]: Uploading segments... Sep 7 04:23:05.779517276 (14h54m41.819090396s) [upload=13122]: Got next segment. Inline: false Sep 7 04:23:05.779550860 (14h54m41.81912398s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 04:23:05.779558204 (14h54m41.819131324s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 04:23:08.191697796 (14h54m44.231270916s) [seg_pos={0 0}, upload=13122]: Begin upload segment... Sep 7 04:23:08.191745878 (14h54m44.231318998s) [seg_pos={0 0}, upload=13122]: Done begin upload segment. Sep 7 04:23:08.191768891 (14h54m44.231342011s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.191773190 (14h54m44.23134631s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192136916 (14h54m44.231710026s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192145131 (14h54m44.231718251s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192406221 (14h54m44.231979331s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192664154 (14h54m44.232237274s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192901128 (14h54m44.232474248s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193209558 (14h54m44.232782678s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193449928 (14h54m44.233023048s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193686511 (14h54m44.233259621s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193897635 (14h54m44.233470755s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194095294 (14h54m44.233668414s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194288964 (14h54m44.233862084s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194473567 (14h54m44.234046687s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194726201 (14h54m44.234299321s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.196014095 (14h54m44.235587215s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:12.514659180 (14h54m48.5542323s) [upload=13122]: Next returned nil segment Sep 7 04:23:12.514677153 (14h54m48.554250273s) [upload=13122]: Waiting for error group managing segments... Sep 7 04:23:14.018528872 (14h54m50.058101992s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.233524174 (14h54m50.273097294s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.455484504 (14h54m50.495057624s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.483549776 (14h54m50.523122896s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.938110455 (14h54m50.977683575s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.992469711 (14h54m51.032042831s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.108041127 (14h54m51.147614247s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.182620684 (14h54m51.222193824s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.392802967 (14h54m51.432376087s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.425736273 (14h54m51.465309393s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.458705710 (14h54m51.49827883s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.738518791 (14h54m51.778091911s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973957025 (14h54m53.013530145s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973983115 (14h54m53.013556235s) [seg_pos={0 0}, upload=13122]: Segment reached optimal threshold of 13 pieces. Sep 7 04:23:16.974131328 (14h54m53.013704438s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Sep 7 04:23:16.974191383 (14h54m53.013764503s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection Sep 7 04:23:16.974224767 (14h54m53.013797887s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection Sep 7 04:23:16.974242441 (14h54m53.013815561s) [seg_pos={0 0}, upload=13122]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 04:23:16.974334678 (14h54m53.013907828s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 04:23:16.974341410 (14h54m53.01391454s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 04:23:17.336870127 (14h54m53.376443257s) [upload=13122]: Done uploading segments... Sep 7 04:23:17.336909021 (14h54m53.376482141s) [upload=13122]: Upload finished. err: <nil> Duration: 6.070748017s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................................X.....XXX..............] duration: 25.457784375s FAIL (2025-09-07 04:44:57 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 25.457809563s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 6.171390045s FAIL (2025-09-07 04:44:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 04:23:05.779258791 (14h54m41.818831921s) [upload=13122]: Starting upload Sep 7 04:23:05.779322874 (14h54m41.818896005s) [upload=13122]: Done starting upload Sep 7 04:23:05.779427754 (14h54m41.819000874s) [upload=13122]: Uploading segments... Sep 7 04:23:05.779517276 (14h54m41.819090396s) [upload=13122]: Got next segment. Inline: false Sep 7 04:23:05.779550860 (14h54m41.81912398s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 04:23:05.779558204 (14h54m41.819131324s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 04:23:08.191697796 (14h54m44.231270916s) [seg_pos={0 0}, upload=13122]: Begin upload segment... Sep 7 04:23:08.191745878 (14h54m44.231318998s) [seg_pos={0 0}, upload=13122]: Done begin upload segment. Sep 7 04:23:08.191768891 (14h54m44.231342011s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.191773190 (14h54m44.23134631s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192136916 (14h54m44.231710026s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192145131 (14h54m44.231718251s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192406221 (14h54m44.231979331s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192664154 (14h54m44.232237274s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192901128 (14h54m44.232474248s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193209558 (14h54m44.232782678s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193449928 (14h54m44.233023048s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193686511 (14h54m44.233259621s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193897635 (14h54m44.233470755s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194095294 (14h54m44.233668414s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194288964 (14h54m44.233862084s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194473567 (14h54m44.234046687s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194726201 (14h54m44.234299321s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.196014095 (14h54m44.235587215s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:12.514659180 (14h54m48.5542323s) [upload=13122]: Next returned nil segment Sep 7 04:23:12.514677153 (14h54m48.554250273s) [upload=13122]: Waiting for error group managing segments... Sep 7 04:23:14.018528872 (14h54m50.058101992s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.233524174 (14h54m50.273097294s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.455484504 (14h54m50.495057624s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.483549776 (14h54m50.523122896s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.938110455 (14h54m50.977683575s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.992469711 (14h54m51.032042831s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.108041127 (14h54m51.147614247s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.182620684 (14h54m51.222193824s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.392802967 (14h54m51.432376087s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.425736273 (14h54m51.465309393s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.458705710 (14h54m51.49827883s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.738518791 (14h54m51.778091911s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973957025 (14h54m53.013530145s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973983115 (14h54m53.013556235s) [seg_pos={0 0}, upload=13122]: Segment reached optimal threshold of 13 pieces. Sep 7 04:23:16.974131328 (14h54m53.013704438s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Sep 7 04:23:16.974191383 (14h54m53.013764503s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection Sep 7 04:23:16.974224767 (14h54m53.013797887s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection Sep 7 04:23:16.974242441 (14h54m53.013815561s) [seg_pos={0 0}, upload=13122]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 04:23:16.974334678 (14h54m53.013907828s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 04:23:16.974341410 (14h54m53.01391454s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 04:23:17.336870127 (14h54m53.376443257s) [upload=13122]: Done uploading segments... Sep 7 04:23:17.336909021 (14h54m53.376482141s) [upload=13122]: Upload finished. err: <nil> Duration: 6.171416195s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.952431787s FAIL (2025-09-07 04:44:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 04:23:05.779258791 (14h54m41.818831921s) [upload=13122]: Starting upload Sep 7 04:23:05.779322874 (14h54m41.818896005s) [upload=13122]: Done starting upload Sep 7 04:23:05.779427754 (14h54m41.819000874s) [upload=13122]: Uploading segments... Sep 7 04:23:05.779517276 (14h54m41.819090396s) [upload=13122]: Got next segment. Inline: false Sep 7 04:23:05.779550860 (14h54m41.81912398s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 04:23:05.779558204 (14h54m41.819131324s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 04:23:08.191697796 (14h54m44.231270916s) [seg_pos={0 0}, upload=13122]: Begin upload segment... Sep 7 04:23:08.191745878 (14h54m44.231318998s) [seg_pos={0 0}, upload=13122]: Done begin upload segment. Sep 7 04:23:08.191768891 (14h54m44.231342011s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.191773190 (14h54m44.23134631s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192136916 (14h54m44.231710026s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192145131 (14h54m44.231718251s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192406221 (14h54m44.231979331s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192664154 (14h54m44.232237274s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192901128 (14h54m44.232474248s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193209558 (14h54m44.232782678s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193449928 (14h54m44.233023048s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193686511 (14h54m44.233259621s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193897635 (14h54m44.233470755s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194095294 (14h54m44.233668414s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194288964 (14h54m44.233862084s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194473567 (14h54m44.234046687s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194726201 (14h54m44.234299321s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.196014095 (14h54m44.235587215s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:12.514659180 (14h54m48.5542323s) [upload=13122]: Next returned nil segment Sep 7 04:23:12.514677153 (14h54m48.554250273s) [upload=13122]: Waiting for error group managing segments... Sep 7 04:23:14.018528872 (14h54m50.058101992s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.233524174 (14h54m50.273097294s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.455484504 (14h54m50.495057624s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.483549776 (14h54m50.523122896s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.938110455 (14h54m50.977683575s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.992469711 (14h54m51.032042831s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.108041127 (14h54m51.147614247s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.182620684 (14h54m51.222193824s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.392802967 (14h54m51.432376087s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.425736273 (14h54m51.465309393s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.458705710 (14h54m51.49827883s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.738518791 (14h54m51.778091911s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973957025 (14h54m53.013530145s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973983115 (14h54m53.013556235s) [seg_pos={0 0}, upload=13122]: Segment reached optimal threshold of 13 pieces. Sep 7 04:23:16.974131328 (14h54m53.013704438s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Sep 7 04:23:16.974191383 (14h54m53.013764503s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection Sep 7 04:23:16.974224767 (14h54m53.013797887s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection Sep 7 04:23:16.974242441 (14h54m53.013815561s) [seg_pos={0 0}, upload=13122]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 04:23:16.974334678 (14h54m53.013907828s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 04:23:16.974341410 (14h54m53.01391454s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 04:23:17.336870127 (14h54m53.376443257s) [upload=13122]: Done uploading segments... Sep 7 04:23:17.336909021 (14h54m53.376482141s) [upload=13122]: Upload finished. err: <nil> Duration: 5.952456977s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........................................X....X............] duration: 1.411929508s FAIL (2025-09-07 04:44:17 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.411969673s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................................X.....XXX.............] duration: 5.261136965s FAIL (2025-09-07 04:43:55 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 5.261167444s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.43007623s FAIL (2025-09-07 04:43:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 04:23:05.779258791 (14h54m41.818831921s) [upload=13122]: Starting upload Sep 7 04:23:05.779322874 (14h54m41.818896005s) [upload=13122]: Done starting upload Sep 7 04:23:05.779427754 (14h54m41.819000874s) [upload=13122]: Uploading segments... Sep 7 04:23:05.779517276 (14h54m41.819090396s) [upload=13122]: Got next segment. Inline: false Sep 7 04:23:05.779550860 (14h54m41.81912398s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 04:23:05.779558204 (14h54m41.819131324s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 04:23:08.191697796 (14h54m44.231270916s) [seg_pos={0 0}, upload=13122]: Begin upload segment... Sep 7 04:23:08.191745878 (14h54m44.231318998s) [seg_pos={0 0}, upload=13122]: Done begin upload segment. Sep 7 04:23:08.191768891 (14h54m44.231342011s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.191773190 (14h54m44.23134631s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192136916 (14h54m44.231710026s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192145131 (14h54m44.231718251s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192406221 (14h54m44.231979331s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192664154 (14h54m44.232237274s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192901128 (14h54m44.232474248s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193209558 (14h54m44.232782678s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193449928 (14h54m44.233023048s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193686511 (14h54m44.233259621s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193897635 (14h54m44.233470755s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194095294 (14h54m44.233668414s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194288964 (14h54m44.233862084s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194473567 (14h54m44.234046687s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194726201 (14h54m44.234299321s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.196014095 (14h54m44.235587215s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:12.514659180 (14h54m48.5542323s) [upload=13122]: Next returned nil segment Sep 7 04:23:12.514677153 (14h54m48.554250273s) [upload=13122]: Waiting for error group managing segments... Sep 7 04:23:14.018528872 (14h54m50.058101992s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.233524174 (14h54m50.273097294s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.455484504 (14h54m50.495057624s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.483549776 (14h54m50.523122896s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.938110455 (14h54m50.977683575s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.992469711 (14h54m51.032042831s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.108041127 (14h54m51.147614247s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.182620684 (14h54m51.222193824s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.392802967 (14h54m51.432376087s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.425736273 (14h54m51.465309393s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.458705710 (14h54m51.49827883s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.738518791 (14h54m51.778091911s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973957025 (14h54m53.013530145s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973983115 (14h54m53.013556235s) [seg_pos={0 0}, upload=13122]: Segment reached optimal threshold of 13 pieces. Sep 7 04:23:16.974131328 (14h54m53.013704438s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Sep 7 04:23:16.974191383 (14h54m53.013764503s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection Sep 7 04:23:16.974224767 (14h54m53.013797887s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection Sep 7 04:23:16.974242441 (14h54m53.013815561s) [seg_pos={0 0}, upload=13122]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 04:23:16.974334678 (14h54m53.013907828s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 04:23:16.974341410 (14h54m53.01391454s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 04:23:17.336870127 (14h54m53.376443257s) [upload=13122]: Done uploading segments... Sep 7 04:23:17.336909021 (14h54m53.376482141s) [upload=13122]: Upload finished. err: <nil> Duration: 6.430104525s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........................................X....X...........] duration: 1.426161932s FAIL (2025-09-07 04:43:30 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.426402933s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................................X.....XXX............] duration: 5.168624412s FAIL (2025-09-07 04:43:07 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 5.168653337s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 2 out of 60 is failed (threshold=3) [............................................X....X..........] duration: 1.376126726s FAIL (2025-09-07 04:42:56 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.376156465s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 6.266680835s FAIL (2025-09-07 04:42:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 04:23:05.779258791 (14h54m41.818831921s) [upload=13122]: Starting upload Sep 7 04:23:05.779322874 (14h54m41.818896005s) [upload=13122]: Done starting upload Sep 7 04:23:05.779427754 (14h54m41.819000874s) [upload=13122]: Uploading segments... Sep 7 04:23:05.779517276 (14h54m41.819090396s) [upload=13122]: Got next segment. Inline: false Sep 7 04:23:05.779550860 (14h54m41.81912398s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 04:23:05.779558204 (14h54m41.819131324s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 04:23:08.191697796 (14h54m44.231270916s) [seg_pos={0 0}, upload=13122]: Begin upload segment... Sep 7 04:23:08.191745878 (14h54m44.231318998s) [seg_pos={0 0}, upload=13122]: Done begin upload segment. Sep 7 04:23:08.191768891 (14h54m44.231342011s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.191773190 (14h54m44.23134631s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192136916 (14h54m44.231710026s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192145131 (14h54m44.231718251s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192406221 (14h54m44.231979331s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192664154 (14h54m44.232237274s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192901128 (14h54m44.232474248s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193209558 (14h54m44.232782678s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193449928 (14h54m44.233023048s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193686511 (14h54m44.233259621s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193897635 (14h54m44.233470755s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194095294 (14h54m44.233668414s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194288964 (14h54m44.233862084s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194473567 (14h54m44.234046687s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194726201 (14h54m44.234299321s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.196014095 (14h54m44.235587215s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:12.514659180 (14h54m48.5542323s) [upload=13122]: Next returned nil segment Sep 7 04:23:12.514677153 (14h54m48.554250273s) [upload=13122]: Waiting for error group managing segments... Sep 7 04:23:14.018528872 (14h54m50.058101992s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.233524174 (14h54m50.273097294s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.455484504 (14h54m50.495057624s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.483549776 (14h54m50.523122896s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.938110455 (14h54m50.977683575s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.992469711 (14h54m51.032042831s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.108041127 (14h54m51.147614247s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.182620684 (14h54m51.222193824s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.392802967 (14h54m51.432376087s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.425736273 (14h54m51.465309393s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.458705710 (14h54m51.49827883s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.738518791 (14h54m51.778091911s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973957025 (14h54m53.013530145s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973983115 (14h54m53.013556235s) [seg_pos={0 0}, upload=13122]: Segment reached optimal threshold of 13 pieces. Sep 7 04:23:16.974131328 (14h54m53.013704438s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Sep 7 04:23:16.974191383 (14h54m53.013764503s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection Sep 7 04:23:16.974224767 (14h54m53.013797887s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection Sep 7 04:23:16.974242441 (14h54m53.013815561s) [seg_pos={0 0}, upload=13122]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 04:23:16.974334678 (14h54m53.013907828s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 04:23:16.974341410 (14h54m53.01391454s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 04:23:17.336870127 (14h54m53.376443257s) [upload=13122]: Done uploading segments... Sep 7 04:23:17.336909021 (14h54m53.376482141s) [upload=13122]: Upload finished. err: <nil> Duration: 6.26670943s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............................................X....X.........] duration: 1.446911856s FAIL (2025-09-07 04:42:23 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.446950499s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................................X.....XXX...........] duration: 5.286153965s FAIL (2025-09-07 04:42:13 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 5.286185746s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.149890936s FAIL (2025-09-07 04:41:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 04:23:05.779258791 (14h54m41.818831921s) [upload=13122]: Starting upload Sep 7 04:23:05.779322874 (14h54m41.818896005s) [upload=13122]: Done starting upload Sep 7 04:23:05.779427754 (14h54m41.819000874s) [upload=13122]: Uploading segments... Sep 7 04:23:05.779517276 (14h54m41.819090396s) [upload=13122]: Got next segment. Inline: false Sep 7 04:23:05.779550860 (14h54m41.81912398s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 04:23:05.779558204 (14h54m41.819131324s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 04:23:08.191697796 (14h54m44.231270916s) [seg_pos={0 0}, upload=13122]: Begin upload segment... Sep 7 04:23:08.191745878 (14h54m44.231318998s) [seg_pos={0 0}, upload=13122]: Done begin upload segment. Sep 7 04:23:08.191768891 (14h54m44.231342011s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.191773190 (14h54m44.23134631s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192136916 (14h54m44.231710026s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192145131 (14h54m44.231718251s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192406221 (14h54m44.231979331s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192664154 (14h54m44.232237274s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192901128 (14h54m44.232474248s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193209558 (14h54m44.232782678s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193449928 (14h54m44.233023048s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193686511 (14h54m44.233259621s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193897635 (14h54m44.233470755s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194095294 (14h54m44.233668414s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194288964 (14h54m44.233862084s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194473567 (14h54m44.234046687s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194726201 (14h54m44.234299321s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.196014095 (14h54m44.235587215s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:12.514659180 (14h54m48.5542323s) [upload=13122]: Next returned nil segment Sep 7 04:23:12.514677153 (14h54m48.554250273s) [upload=13122]: Waiting for error group managing segments... Sep 7 04:23:14.018528872 (14h54m50.058101992s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.233524174 (14h54m50.273097294s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.455484504 (14h54m50.495057624s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.483549776 (14h54m50.523122896s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.938110455 (14h54m50.977683575s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.992469711 (14h54m51.032042831s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.108041127 (14h54m51.147614247s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.182620684 (14h54m51.222193824s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.392802967 (14h54m51.432376087s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.425736273 (14h54m51.465309393s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.458705710 (14h54m51.49827883s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.738518791 (14h54m51.778091911s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973957025 (14h54m53.013530145s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973983115 (14h54m53.013556235s) [seg_pos={0 0}, upload=13122]: Segment reached optimal threshold of 13 pieces. Sep 7 04:23:16.974131328 (14h54m53.013704438s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Sep 7 04:23:16.974191383 (14h54m53.013764503s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection Sep 7 04:23:16.974224767 (14h54m53.013797887s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection Sep 7 04:23:16.974242441 (14h54m53.013815561s) [seg_pos={0 0}, upload=13122]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 04:23:16.974334678 (14h54m53.013907828s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 04:23:16.974341410 (14h54m53.01391454s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 04:23:17.336870127 (14h54m53.376443257s) [upload=13122]: Done uploading segments... Sep 7 04:23:17.336909021 (14h54m53.376482141s) [upload=13122]: Upload finished. err: <nil> Duration: 6.149915562s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............................................X....X........] duration: 1.392748274s FAIL (2025-09-07 04:41:52 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.392787529s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........................................X.....XXX..........] duration: 5.116489837s FAIL (2025-09-07 04:41:36 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 5.116523041s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........................................X.....XXX.........] duration: 5.17011334s FAIL (2025-09-07 04:41:18 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 5.170141494s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.288256892s FAIL (2025-09-07 04:41:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 04:23:05.779258791 (14h54m41.818831921s) [upload=13122]: Starting upload Sep 7 04:23:05.779322874 (14h54m41.818896005s) [upload=13122]: Done starting upload Sep 7 04:23:05.779427754 (14h54m41.819000874s) [upload=13122]: Uploading segments... Sep 7 04:23:05.779517276 (14h54m41.819090396s) [upload=13122]: Got next segment. Inline: false Sep 7 04:23:05.779550860 (14h54m41.81912398s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 04:23:05.779558204 (14h54m41.819131324s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 04:23:08.191697796 (14h54m44.231270916s) [seg_pos={0 0}, upload=13122]: Begin upload segment... Sep 7 04:23:08.191745878 (14h54m44.231318998s) [seg_pos={0 0}, upload=13122]: Done begin upload segment. Sep 7 04:23:08.191768891 (14h54m44.231342011s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.191773190 (14h54m44.23134631s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192136916 (14h54m44.231710026s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192145131 (14h54m44.231718251s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192406221 (14h54m44.231979331s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192664154 (14h54m44.232237274s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192901128 (14h54m44.232474248s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193209558 (14h54m44.232782678s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193449928 (14h54m44.233023048s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193686511 (14h54m44.233259621s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193897635 (14h54m44.233470755s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194095294 (14h54m44.233668414s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194288964 (14h54m44.233862084s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194473567 (14h54m44.234046687s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194726201 (14h54m44.234299321s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.196014095 (14h54m44.235587215s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:12.514659180 (14h54m48.5542323s) [upload=13122]: Next returned nil segment Sep 7 04:23:12.514677153 (14h54m48.554250273s) [upload=13122]: Waiting for error group managing segments... Sep 7 04:23:14.018528872 (14h54m50.058101992s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.233524174 (14h54m50.273097294s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.455484504 (14h54m50.495057624s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.483549776 (14h54m50.523122896s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.938110455 (14h54m50.977683575s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.992469711 (14h54m51.032042831s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.108041127 (14h54m51.147614247s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.182620684 (14h54m51.222193824s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.392802967 (14h54m51.432376087s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.425736273 (14h54m51.465309393s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.458705710 (14h54m51.49827883s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.738518791 (14h54m51.778091911s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973957025 (14h54m53.013530145s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973983115 (14h54m53.013556235s) [seg_pos={0 0}, upload=13122]: Segment reached optimal threshold of 13 pieces. Sep 7 04:23:16.974131328 (14h54m53.013704438s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Sep 7 04:23:16.974191383 (14h54m53.013764503s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection Sep 7 04:23:16.974224767 (14h54m53.013797887s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection Sep 7 04:23:16.974242441 (14h54m53.013815561s) [seg_pos={0 0}, upload=13122]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 04:23:16.974334678 (14h54m53.013907828s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 04:23:16.974341410 (14h54m53.01391454s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 04:23:17.336870127 (14h54m53.376443257s) [upload=13122]: Done uploading segments... Sep 7 04:23:17.336909021 (14h54m53.376482141s) [upload=13122]: Upload finished. err: <nil> Duration: 6.288301748s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........................................X.....XXX........] duration: 5.190052466s FAIL (2025-09-07 04:40:52 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 5.190112863s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............................................X....X.......] duration: 1.560411979s FAIL (2025-09-07 04:40:45 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.560431657s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 2 out of 60 is failed (threshold=3) [................................................X....X......] duration: 22.350355679s FAIL (2025-09-07 04:40:43 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 22.350381267s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 6.073919746s FAIL (2025-09-07 04:40:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 04:23:05.779258791 (14h54m41.818831921s) [upload=13122]: Starting upload Sep 7 04:23:05.779322874 (14h54m41.818896005s) [upload=13122]: Done starting upload Sep 7 04:23:05.779427754 (14h54m41.819000874s) [upload=13122]: Uploading segments... Sep 7 04:23:05.779517276 (14h54m41.819090396s) [upload=13122]: Got next segment. Inline: false Sep 7 04:23:05.779550860 (14h54m41.81912398s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 04:23:05.779558204 (14h54m41.819131324s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 04:23:08.191697796 (14h54m44.231270916s) [seg_pos={0 0}, upload=13122]: Begin upload segment... Sep 7 04:23:08.191745878 (14h54m44.231318998s) [seg_pos={0 0}, upload=13122]: Done begin upload segment. Sep 7 04:23:08.191768891 (14h54m44.231342011s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.191773190 (14h54m44.23134631s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192136916 (14h54m44.231710026s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192145131 (14h54m44.231718251s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192406221 (14h54m44.231979331s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192664154 (14h54m44.232237274s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192901128 (14h54m44.232474248s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193209558 (14h54m44.232782678s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193449928 (14h54m44.233023048s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193686511 (14h54m44.233259621s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193897635 (14h54m44.233470755s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194095294 (14h54m44.233668414s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194288964 (14h54m44.233862084s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194473567 (14h54m44.234046687s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194726201 (14h54m44.234299321s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.196014095 (14h54m44.235587215s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:12.514659180 (14h54m48.5542323s) [upload=13122]: Next returned nil segment Sep 7 04:23:12.514677153 (14h54m48.554250273s) [upload=13122]: Waiting for error group managing segments... Sep 7 04:23:14.018528872 (14h54m50.058101992s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.233524174 (14h54m50.273097294s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.455484504 (14h54m50.495057624s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.483549776 (14h54m50.523122896s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.938110455 (14h54m50.977683575s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.992469711 (14h54m51.032042831s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.108041127 (14h54m51.147614247s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.182620684 (14h54m51.222193824s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.392802967 (14h54m51.432376087s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.425736273 (14h54m51.465309393s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.458705710 (14h54m51.49827883s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.738518791 (14h54m51.778091911s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973957025 (14h54m53.013530145s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973983115 (14h54m53.013556235s) [seg_pos={0 0}, upload=13122]: Segment reached optimal threshold of 13 pieces. Sep 7 04:23:16.974131328 (14h54m53.013704438s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Sep 7 04:23:16.974191383 (14h54m53.013764503s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection Sep 7 04:23:16.974224767 (14h54m53.013797887s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection Sep 7 04:23:16.974242441 (14h54m53.013815561s) [seg_pos={0 0}, upload=13122]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 04:23:16.974334678 (14h54m53.013907828s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 04:23:16.974341410 (14h54m53.01391454s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 04:23:17.336870127 (14h54m53.376443257s) [upload=13122]: Done uploading segments... Sep 7 04:23:17.336909021 (14h54m53.376482141s) [upload=13122]: Upload finished. err: <nil> Duration: 6.073947759s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.354300403s FAIL (2025-09-07 04:40:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 04:23:05.779258791 (14h54m41.818831921s) [upload=13122]: Starting upload Sep 7 04:23:05.779322874 (14h54m41.818896005s) [upload=13122]: Done starting upload Sep 7 04:23:05.779427754 (14h54m41.819000874s) [upload=13122]: Uploading segments... Sep 7 04:23:05.779517276 (14h54m41.819090396s) [upload=13122]: Got next segment. Inline: false Sep 7 04:23:05.779550860 (14h54m41.81912398s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 04:23:05.779558204 (14h54m41.819131324s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 04:23:08.191697796 (14h54m44.231270916s) [seg_pos={0 0}, upload=13122]: Begin upload segment... Sep 7 04:23:08.191745878 (14h54m44.231318998s) [seg_pos={0 0}, upload=13122]: Done begin upload segment. Sep 7 04:23:08.191768891 (14h54m44.231342011s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.191773190 (14h54m44.23134631s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192136916 (14h54m44.231710026s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192145131 (14h54m44.231718251s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192406221 (14h54m44.231979331s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192664154 (14h54m44.232237274s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192901128 (14h54m44.232474248s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193209558 (14h54m44.232782678s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193449928 (14h54m44.233023048s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193686511 (14h54m44.233259621s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193897635 (14h54m44.233470755s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194095294 (14h54m44.233668414s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194288964 (14h54m44.233862084s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194473567 (14h54m44.234046687s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194726201 (14h54m44.234299321s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.196014095 (14h54m44.235587215s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:12.514659180 (14h54m48.5542323s) [upload=13122]: Next returned nil segment Sep 7 04:23:12.514677153 (14h54m48.554250273s) [upload=13122]: Waiting for error group managing segments... Sep 7 04:23:14.018528872 (14h54m50.058101992s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.233524174 (14h54m50.273097294s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.455484504 (14h54m50.495057624s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.483549776 (14h54m50.523122896s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.938110455 (14h54m50.977683575s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.992469711 (14h54m51.032042831s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.108041127 (14h54m51.147614247s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.182620684 (14h54m51.222193824s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.392802967 (14h54m51.432376087s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.425736273 (14h54m51.465309393s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.458705710 (14h54m51.49827883s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.738518791 (14h54m51.778091911s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973957025 (14h54m53.013530145s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973983115 (14h54m53.013556235s) [seg_pos={0 0}, upload=13122]: Segment reached optimal threshold of 13 pieces. Sep 7 04:23:16.974131328 (14h54m53.013704438s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Sep 7 04:23:16.974191383 (14h54m53.013764503s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection Sep 7 04:23:16.974224767 (14h54m53.013797887s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection Sep 7 04:23:16.974242441 (14h54m53.013815561s) [seg_pos={0 0}, upload=13122]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 04:23:16.974334678 (14h54m53.013907828s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 04:23:16.974341410 (14h54m53.01391454s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 04:23:17.336870127 (14h54m53.376443257s) [upload=13122]: Done uploading segments... Sep 7 04:23:17.336909021 (14h54m53.376482141s) [upload=13122]: Upload finished. err: <nil> Duration: 6.35432031s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 4 out of 60 is failed (threshold=3) [............................................X.....XXX.......] duration: 5.114905998s FAIL (2025-09-07 04:40:07 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 5.11493345s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.................................................X....X.....] duration: 1.365301896s FAIL (2025-09-07 04:39:53 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.365325922s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 7.362138223s FAIL (2025-09-07 04:39:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 04:23:05.779258791 (14h54m41.818831921s) [upload=13122]: Starting upload Sep 7 04:23:05.779322874 (14h54m41.818896005s) [upload=13122]: Done starting upload Sep 7 04:23:05.779427754 (14h54m41.819000874s) [upload=13122]: Uploading segments... Sep 7 04:23:05.779517276 (14h54m41.819090396s) [upload=13122]: Got next segment. Inline: false Sep 7 04:23:05.779550860 (14h54m41.81912398s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 04:23:05.779558204 (14h54m41.819131324s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 04:23:08.191697796 (14h54m44.231270916s) [seg_pos={0 0}, upload=13122]: Begin upload segment... Sep 7 04:23:08.191745878 (14h54m44.231318998s) [seg_pos={0 0}, upload=13122]: Done begin upload segment. Sep 7 04:23:08.191768891 (14h54m44.231342011s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.191773190 (14h54m44.23134631s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192136916 (14h54m44.231710026s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192145131 (14h54m44.231718251s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192406221 (14h54m44.231979331s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192664154 (14h54m44.232237274s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192901128 (14h54m44.232474248s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193209558 (14h54m44.232782678s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193449928 (14h54m44.233023048s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193686511 (14h54m44.233259621s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193897635 (14h54m44.233470755s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194095294 (14h54m44.233668414s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194288964 (14h54m44.233862084s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194473567 (14h54m44.234046687s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194726201 (14h54m44.234299321s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.196014095 (14h54m44.235587215s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:12.514659180 (14h54m48.5542323s) [upload=13122]: Next returned nil segment Sep 7 04:23:12.514677153 (14h54m48.554250273s) [upload=13122]: Waiting for error group managing segments... Sep 7 04:23:14.018528872 (14h54m50.058101992s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.233524174 (14h54m50.273097294s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.455484504 (14h54m50.495057624s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.483549776 (14h54m50.523122896s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.938110455 (14h54m50.977683575s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.992469711 (14h54m51.032042831s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.108041127 (14h54m51.147614247s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.182620684 (14h54m51.222193824s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.392802967 (14h54m51.432376087s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.425736273 (14h54m51.465309393s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.458705710 (14h54m51.49827883s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.738518791 (14h54m51.778091911s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973957025 (14h54m53.013530145s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973983115 (14h54m53.013556235s) [seg_pos={0 0}, upload=13122]: Segment reached optimal threshold of 13 pieces. Sep 7 04:23:16.974131328 (14h54m53.013704438s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Sep 7 04:23:16.974191383 (14h54m53.013764503s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection Sep 7 04:23:16.974224767 (14h54m53.013797887s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection Sep 7 04:23:16.974242441 (14h54m53.013815561s) [seg_pos={0 0}, upload=13122]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 04:23:16.974334678 (14h54m53.013907828s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 04:23:16.974341410 (14h54m53.01391454s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 04:23:17.336870127 (14h54m53.376443257s) [upload=13122]: Done uploading segments... Sep 7 04:23:17.336909021 (14h54m53.376482141s) [upload=13122]: Upload finished. err: <nil> Duration: 7.362162148s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............................................X.....XXX......] duration: 5.077020465s FAIL (2025-09-07 04:39:42 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 5.077054981s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 6.194022363s FAIL (2025-09-07 04:39:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 04:23:05.779258791 (14h54m41.818831921s) [upload=13122]: Starting upload Sep 7 04:23:05.779322874 (14h54m41.818896005s) [upload=13122]: Done starting upload Sep 7 04:23:05.779427754 (14h54m41.819000874s) [upload=13122]: Uploading segments... Sep 7 04:23:05.779517276 (14h54m41.819090396s) [upload=13122]: Got next segment. Inline: false Sep 7 04:23:05.779550860 (14h54m41.81912398s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 04:23:05.779558204 (14h54m41.819131324s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 04:23:08.191697796 (14h54m44.231270916s) [seg_pos={0 0}, upload=13122]: Begin upload segment... Sep 7 04:23:08.191745878 (14h54m44.231318998s) [seg_pos={0 0}, upload=13122]: Done begin upload segment. Sep 7 04:23:08.191768891 (14h54m44.231342011s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.191773190 (14h54m44.23134631s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192136916 (14h54m44.231710026s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192145131 (14h54m44.231718251s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192406221 (14h54m44.231979331s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192664154 (14h54m44.232237274s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192901128 (14h54m44.232474248s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193209558 (14h54m44.232782678s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193449928 (14h54m44.233023048s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193686511 (14h54m44.233259621s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193897635 (14h54m44.233470755s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194095294 (14h54m44.233668414s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194288964 (14h54m44.233862084s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194473567 (14h54m44.234046687s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194726201 (14h54m44.234299321s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.196014095 (14h54m44.235587215s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:12.514659180 (14h54m48.5542323s) [upload=13122]: Next returned nil segment Sep 7 04:23:12.514677153 (14h54m48.554250273s) [upload=13122]: Waiting for error group managing segments... Sep 7 04:23:14.018528872 (14h54m50.058101992s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.233524174 (14h54m50.273097294s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.455484504 (14h54m50.495057624s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.483549776 (14h54m50.523122896s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.938110455 (14h54m50.977683575s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.992469711 (14h54m51.032042831s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.108041127 (14h54m51.147614247s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.182620684 (14h54m51.222193824s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.392802967 (14h54m51.432376087s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.425736273 (14h54m51.465309393s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.458705710 (14h54m51.49827883s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.738518791 (14h54m51.778091911s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973957025 (14h54m53.013530145s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973983115 (14h54m53.013556235s) [seg_pos={0 0}, upload=13122]: Segment reached optimal threshold of 13 pieces. Sep 7 04:23:16.974131328 (14h54m53.013704438s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Sep 7 04:23:16.974191383 (14h54m53.013764503s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection Sep 7 04:23:16.974224767 (14h54m53.013797887s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection Sep 7 04:23:16.974242441 (14h54m53.013815561s) [seg_pos={0 0}, upload=13122]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 04:23:16.974334678 (14h54m53.013907828s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 04:23:16.974341410 (14h54m53.01391454s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 04:23:17.336870127 (14h54m53.376443257s) [upload=13122]: Done uploading segments... Sep 7 04:23:17.336909021 (14h54m53.376482141s) [upload=13122]: Upload finished. err: <nil> Duration: 6.194074492s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 2 out of 60 is failed (threshold=3) [..................................................X....X....] duration: 1.409610991s FAIL (2025-09-07 04:39:04 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.40964062s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 6.251168162s FAIL (2025-09-07 04:38:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 7 04:23:05.779258791 (14h54m41.818831921s) [upload=13122]: Starting upload Sep 7 04:23:05.779322874 (14h54m41.818896005s) [upload=13122]: Done starting upload Sep 7 04:23:05.779427754 (14h54m41.819000874s) [upload=13122]: Uploading segments... Sep 7 04:23:05.779517276 (14h54m41.819090396s) [upload=13122]: Got next segment. Inline: false Sep 7 04:23:05.779550860 (14h54m41.81912398s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 7 04:23:05.779558204 (14h54m41.819131324s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 7 04:23:08.191697796 (14h54m44.231270916s) [seg_pos={0 0}, upload=13122]: Begin upload segment... Sep 7 04:23:08.191745878 (14h54m44.231318998s) [seg_pos={0 0}, upload=13122]: Done begin upload segment. Sep 7 04:23:08.191768891 (14h54m44.231342011s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.191773190 (14h54m44.23134631s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192136916 (14h54m44.231710026s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192145131 (14h54m44.231718251s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192406221 (14h54m44.231979331s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192664154 (14h54m44.232237274s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.192901128 (14h54m44.232474248s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193209558 (14h54m44.232782678s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193449928 (14h54m44.233023048s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193686511 (14h54m44.233259621s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.193897635 (14h54m44.233470755s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194095294 (14h54m44.233668414s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194288964 (14h54m44.233862084s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194473567 (14h54m44.234046687s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.194726201 (14h54m44.234299321s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:08.196014095 (14h54m44.235587215s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Uploading piece... Sep 7 04:23:12.514659180 (14h54m48.5542323s) [upload=13122]: Next returned nil segment Sep 7 04:23:12.514677153 (14h54m48.554250273s) [upload=13122]: Waiting for error group managing segments... Sep 7 04:23:14.018528872 (14h54m50.058101992s) [noise=true , address=45.91.116.12:4227 , piece_id=QQ4G43PKIXMW5XNBMFITDXO2YOD3IXXOV2RSJGA5SRSXVI5BS6JQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.233524174 (14h54m50.273097294s) [noise=true , address=45.91.116.5:4298 , piece_id=247OU22BPXW3IVZUWYGBGV4F2WHYJ4LVBPAISQZITGZMFJLHIEWA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.455484504 (14h54m50.495057624s) [noise=true , address=45.91.116.4:4939 , piece_id=7IHTMYERJG3A77J6VNBDADXKAWVRLTST5F77VSJDEZFTMRFKSCXA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.483549776 (14h54m50.523122896s) [noise=true , address=45.91.116.13:4153 , piece_id=ADK2X6XV7FHMUHZ5MAIMVY7HUHH7MM3WDYH2KKKJOCF7BH33HFSA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.938110455 (14h54m50.977683575s) [noise=true , address=45.91.116.8:4047 , piece_id=BNR3TQ2BMBCNWKPFZJLVJS2EYPRW7PW5VC542CEUYQRN3KV7CAUA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:14.992469711 (14h54m51.032042831s) [noise=true , address=45.91.116.10:4082 , piece_id=SGY4QTUAMLG5D6U75FSIL5C2TJ5LGK7DLZDQZ4ZQTJRLT4F7PWZQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.108041127 (14h54m51.147614247s) [noise=true , address=77.90.31.15:4699 , piece_id=V3GVHDBTVCQ3JNLIO2WF7VI2YCXQHHHENXTJGAIBHF3XXJCF7IQA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.182620684 (14h54m51.222193824s) [noise=true , address=77.90.31.12:4592 , piece_id=6C35H6HMZURZIW5N4FDN7BE2XQLVK7K5V7JLYXZCFVHIGL4RCFCQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.392802967 (14h54m51.432376087s) [noise=true , address=45.91.116.11:4118 , piece_id=33BGUTO7TKFTYYAA5X4RNW4BPGJG3BKHTZERSSZFMO2DCMP5MUVQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.425736273 (14h54m51.465309393s) [noise=true , address=77.90.31.10:4520 , piece_id=FQDC5SEGGD54HL6QSBY26MJDKNMGLGLBA4ICITEBWA5ZJOECS6DA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.458705710 (14h54m51.49827883s) [noise=true , address=45.91.116.13:4155 , piece_id=ZEPLCOSXMPQN74HKR7D3EX77WYH7HSNMEGAU4CP7AK4A5F3OUDHA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:15.738518791 (14h54m51.778091911s) [noise=true , address=45.91.116.10:4083 , piece_id=MT7PG436UA35UQOYQWC573URQDV3EZ4VNWKUBO4OVVKGLLAVKOKQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973957025 (14h54m53.013530145s) [noise=true , address=45.91.116.8:4045 , piece_id=LRRCJY6MUY4H5OSOZBH5RF5Y2XLLFWJA2EHHSL7OMG43UESRBLGA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: <nil> Sep 7 04:23:16.973983115 (14h54m53.013556235s) [seg_pos={0 0}, upload=13122]: Segment reached optimal threshold of 13 pieces. Sep 7 04:23:16.974131328 (14h54m53.013704438s) [noise=true , address=45.91.116.4:4937 , piece_id=UCKW4LSEOM2XQM72UTQKF6TBQXW2PJS2JQIIOVX7HRY56JCBMP7A, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Sep 7 04:23:16.974191383 (14h54m53.013764503s) [noise=true , address=45.91.116.8:4048 , piece_id=KX5ZOSTD5AJGOGUZHPJBPZXTVOTBUEI5DELWZG2PMNVNH6QGQYJA, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40884->45.91.116.8:4048: use of closed network connection Sep 7 04:23:16.974224767 (14h54m53.013797887s) [noise=true , address=45.91.116.6:4973 , piece_id=2IIAHN26L4X34DHTYEZLMGDGRMUIXGMIDILMCOVKP6RZVSLGOAXQ, seg_pos={0 0}, upload=13122]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60612->45.91.116.6:4973: use of closed network connection Sep 7 04:23:16.974242441 (14h54m53.013815561s) [seg_pos={0 0}, upload=13122]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 7 04:23:16.974334678 (14h54m53.013907828s) [upload=13122]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 7 04:23:16.974341410 (14h54m53.01391454s) [upload=13122]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 7 04:23:17.336870127 (14h54m53.376443257s) [upload=13122]: Done uploading segments... Sep 7 04:23:17.336909021 (14h54m53.376482141s) [upload=13122]: Upload finished. err: <nil> Duration: 6.251212948s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 2 out of 60 is failed (threshold=3) [...................................................X....X...] duration: 1.418887533s FAIL (2025-09-07 04:38:42 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 1.418913382s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............................................X.....XXX.....] duration: 5.034903625s FAIL (2025-09-07 04:38:37 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 5.034925417s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............................................X.....XXX....] duration: 30.003774579s FAIL (2025-09-07 04:38:32 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 30.003802373s core uplink eu1

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)