Orbiter History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.095886861s FAIL (2025-08-26 09:15:33 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 5.095915915s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................................X...X....] duration: 12.700487904s FAIL (2025-08-26 09:15:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 09:12:27.460801938 (17h8m54.619567672s) [upload=14994]: Starting upload Aug 26 09:12:27.460861261 (17h8m54.619626995s) [upload=14994]: Done starting upload Aug 26 09:12:27.460967244 (17h8m54.619732978s) [upload=14994]: Uploading segments... Aug 26 09:12:27.461061044 (17h8m54.619826768s) [upload=14994]: Got next segment. Inline: false Aug 26 09:12:27.461137420 (17h8m54.619903154s) [upload=14994]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 09:12:27.461142410 (17h8m54.619908144s) [upload=14994]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 09:12:27.911433899 (17h8m55.070199633s) [seg_pos={0 0}, upload=14994]: Begin upload segment... Aug 26 09:12:27.911481470 (17h8m55.070247204s) [seg_pos={0 0}, upload=14994]: Done begin upload segment. Aug 26 09:12:27.911501920 (17h8m55.070267654s) [noise=true , address=104.36.82.7:4912 , piece_id=OYVJF5VJJMZGE6E2JREPMGIZGWKQQWAHK4WCUHXFATZZGJXFJW2Q, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.911554130 (17h8m55.070319904s) [noise=true , address=45.91.116.14:4191 , piece_id=YO65KQXJOPP2BVDRQTEURGV2RDY3M36W4JSPPBTM6LKVHZ7UIEXA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.911868270 (17h8m55.070634004s) [noise=true , address=45.91.116.11:4119 , piece_id=RRRPLNZBTVHIQTUMONDVTIBCMEBLBM2RPFCOHX7RBX4HFGZUYS3Q, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912181570 (17h8m55.070947305s) [noise=true , address=45.91.116.10:4081 , piece_id=YYCAJYYQ4MXTVLBHWA6R66TIAHAEJOUQ7BBH6A3PVB56MCXUXCDA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912244641 (17h8m55.071010365s) [noise=true , address=77.90.31.16:4735 , piece_id=F5ZXRAIRT3GGYVDCOHQWEAWDTJ5DN2OF2AW5YB3W3MTHTUUTVJ6Q, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912475152 (17h8m55.071240886s) [noise=true , address=45.91.116.5:4299 , piece_id=G2JPCAKEMRINSSYZ24B362KA54Q2T6HDEGJQFHVWNEJGGFGRR2KA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912684673 (17h8m55.071450407s) [noise=true , address=45.91.116.9:4262 , piece_id=WRDGDP5TBAYZTRPBYDW4ZESCR7NJJHVBU6WP4UYC4B3V5D7ICOMA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912718718 (17h8m55.071484452s) [noise=true , address=77.90.31.18:4806 , piece_id=RRTHOAKZSK6ZKWLXCMODWX5HZ5I62OMUEBB64FS6RSDLMNL2OESA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912890155 (17h8m55.071655889s) [noise=true , address=45.91.116.4:4937 , piece_id=AZQ6DES5FGD63E56HNPDEOJJUZNKOF3ZYU33OMOUX47L7ZGYXISQ, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912973765 (17h8m55.071739499s) [noise=true , address=104.36.82.4:4802 , piece_id=6TH5NRIXI5GLRDDZDKZJZWPQYCK7BTMF2U5TDQYOV3SL227M6IZA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913095419 (17h8m55.071861153s) [noise=true , address=104.36.82.10:4079 , piece_id=RSWLIXFVVRXKXD6TI4TDKEPW3HY6XU5AUAAH5N3X4UIFNEU3H5CQ, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913202434 (17h8m55.071968168s) [noise=true , address=77.90.31.15:4697 , piece_id=VLJCK73RVDCWWPAC7HUVHLQV2AYUPRYNEA7SNSO7P2ZMRK2FOLSQ, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913320449 (17h8m55.072086183s) [noise=true , address=45.91.116.9:4263 , piece_id=RYI4SAUA6IXKXAAHC62LDOKQUKQNTEG7QCG4QHUGMSREVGWEMNGA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913452622 (17h8m55.072218356s) [noise=true , address=77.90.31.12:4592 , piece_id=4RLI6WE2HS6N2QRLJQZXSYG65EIMVROLJVYNW56CSY254ZLREUYA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913534449 (17h8m55.072300183s) [noise=true , address=104.36.82.8:4971 , piece_id=UZT2NT7JR4F7NIXPSY2ORFBQVJOXZSPEJCSLFAHPH4BDSLY2W3DA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913650701 (17h8m55.072416435s) [noise=true , address=77.90.31.12:4589 , piece_id=L6ZTZ5PJPV2WM2BZGGAJEKBS4B2TS5S55PSD7WGA2YRH3MTLG5GA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:29.900914740 (17h8m57.059680464s) [upload=14994]: Next returned nil segment Aug 26 09:12:29.900934336 (17h8m57.059700081s) [upload=14994]: Waiting for error group managing segments... Aug 26 09:12:30.221707715 (17h8m57.380473449s) [noise=true , address=77.90.31.18:4806 , piece_id=RRTHOAKZSK6ZKWLXCMODWX5HZ5I62OMUEBB64FS6RSDLMNL2OESA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.225253190 (17h8m57.384018924s) [noise=true , address=77.90.31.12:4592 , piece_id=4RLI6WE2HS6N2QRLJQZXSYG65EIMVROLJVYNW56CSY254ZLREUYA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.228153169 (17h8m57.386918893s) [noise=true , address=45.91.116.4:4937 , piece_id=AZQ6DES5FGD63E56HNPDEOJJUZNKOF3ZYU33OMOUX47L7ZGYXISQ, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.244451243 (17h8m57.403216977s) [noise=true , address=45.91.116.9:4263 , piece_id=RYI4SAUA6IXKXAAHC62LDOKQUKQNTEG7QCG4QHUGMSREVGWEMNGA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.274597415 (17h8m57.433363149s) [noise=true , address=45.91.116.5:4299 , piece_id=G2JPCAKEMRINSSYZ24B362KA54Q2T6HDEGJQFHVWNEJGGFGRR2KA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.280700783 (17h8m57.439466517s) [noise=true , address=104.36.82.4:4802 , piece_id=6TH5NRIXI5GLRDDZDKZJZWPQYCK7BTMF2U5TDQYOV3SL227M6IZA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.294073090 (17h8m57.452838824s) [noise=true , address=104.36.82.7:4912 , piece_id=OYVJF5VJJMZGE6E2JREPMGIZGWKQQWAHK4WCUHXFATZZGJXFJW2Q, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.310065069 (17h8m57.468830803s) [noise=true , address=104.36.82.8:4971 , piece_id=UZT2NT7JR4F7NIXPSY2ORFBQVJOXZSPEJCSLFAHPH4BDSLY2W3DA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.310725412 (17h8m57.469491146s) [noise=true , address=45.91.116.10:4081 , piece_id=YYCAJYYQ4MXTVLBHWA6R66TIAHAEJOUQ7BBH6A3PVB56MCXUXCDA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.327312570 (17h8m57.486078304s) [noise=true , address=77.90.31.15:4697 , piece_id=VLJCK73RVDCWWPAC7HUVHLQV2AYUPRYNEA7SNSO7P2ZMRK2FOLSQ, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.371703187 (17h8m57.530468921s) [noise=true , address=77.90.31.16:4735 , piece_id=F5ZXRAIRT3GGYVDCOHQWEAWDTJ5DN2OF2AW5YB3W3MTHTUUTVJ6Q, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.427020261 (17h8m57.585785996s) [noise=true , address=45.91.116.11:4119 , piece_id=RRRPLNZBTVHIQTUMONDVTIBCMEBLBM2RPFCOHX7RBX4HFGZUYS3Q, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.436406982 (17h8m57.595172716s) [noise=true , address=45.91.116.14:4191 , piece_id=YO65KQXJOPP2BVDRQTEURGV2RDY3M36W4JSPPBTM6LKVHZ7UIEXA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.436435246 (17h8m57.59520098s) [seg_pos={0 0}, upload=14994]: Segment reached optimal threshold of 13 pieces. Aug 26 09:12:30.436615631 (17h8m57.595381365s) [noise=true , address=77.90.31.12:4589 , piece_id=L6ZTZ5PJPV2WM2BZGGAJEKBS4B2TS5S55PSD7WGA2YRH3MTLG5GA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Aug 26 09:12:30.436693600 (17h8m57.595459334s) [noise=true , address=45.91.116.9:4262 , piece_id=WRDGDP5TBAYZTRPBYDW4ZESCR7NJJHVBU6WP4UYC4B3V5D7ICOMA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: write tcp 188.166.225.170:55590->45.91.116.9:4262: use of closed network connection; write tcp 188.166.225.170:55590->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55590->45.91.116.9:4262: use of closed network connection Aug 26 09:12:30.436700754 (17h8m57.595466488s) [noise=true , address=104.36.82.10:4079 , piece_id=RSWLIXFVVRXKXD6TI4TDKEPW3HY6XU5AUAAH5N3X4UIFNEU3H5CQ, seg_pos={0 0}, upload=14994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Aug 26 09:12:30.436750138 (17h8m57.595515872s) [seg_pos={0 0}, upload=14994]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 09:12:30.436849909 (17h8m57.595615643s) [upload=14994]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 09:12:30.436855740 (17h8m57.595621474s) [upload=14994]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 09:12:30.739827348 (17h8m57.898593082s) [upload=14994]: Done uploading segments... Aug 26 09:12:30.739846063 (17h8m57.898611797s) [upload=14994]: Upload finished. err: <nil> Duration: 12.700540796s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................................X...X...] duration: 8.414417291s FAIL (2025-08-26 09:14:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 09:12:27.460801938 (17h8m54.619567672s) [upload=14994]: Starting upload Aug 26 09:12:27.460861261 (17h8m54.619626995s) [upload=14994]: Done starting upload Aug 26 09:12:27.460967244 (17h8m54.619732978s) [upload=14994]: Uploading segments... Aug 26 09:12:27.461061044 (17h8m54.619826768s) [upload=14994]: Got next segment. Inline: false Aug 26 09:12:27.461137420 (17h8m54.619903154s) [upload=14994]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 09:12:27.461142410 (17h8m54.619908144s) [upload=14994]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 09:12:27.911433899 (17h8m55.070199633s) [seg_pos={0 0}, upload=14994]: Begin upload segment... Aug 26 09:12:27.911481470 (17h8m55.070247204s) [seg_pos={0 0}, upload=14994]: Done begin upload segment. Aug 26 09:12:27.911501920 (17h8m55.070267654s) [noise=true , address=104.36.82.7:4912 , piece_id=OYVJF5VJJMZGE6E2JREPMGIZGWKQQWAHK4WCUHXFATZZGJXFJW2Q, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.911554130 (17h8m55.070319904s) [noise=true , address=45.91.116.14:4191 , piece_id=YO65KQXJOPP2BVDRQTEURGV2RDY3M36W4JSPPBTM6LKVHZ7UIEXA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.911868270 (17h8m55.070634004s) [noise=true , address=45.91.116.11:4119 , piece_id=RRRPLNZBTVHIQTUMONDVTIBCMEBLBM2RPFCOHX7RBX4HFGZUYS3Q, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912181570 (17h8m55.070947305s) [noise=true , address=45.91.116.10:4081 , piece_id=YYCAJYYQ4MXTVLBHWA6R66TIAHAEJOUQ7BBH6A3PVB56MCXUXCDA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912244641 (17h8m55.071010365s) [noise=true , address=77.90.31.16:4735 , piece_id=F5ZXRAIRT3GGYVDCOHQWEAWDTJ5DN2OF2AW5YB3W3MTHTUUTVJ6Q, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912475152 (17h8m55.071240886s) [noise=true , address=45.91.116.5:4299 , piece_id=G2JPCAKEMRINSSYZ24B362KA54Q2T6HDEGJQFHVWNEJGGFGRR2KA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912684673 (17h8m55.071450407s) [noise=true , address=45.91.116.9:4262 , piece_id=WRDGDP5TBAYZTRPBYDW4ZESCR7NJJHVBU6WP4UYC4B3V5D7ICOMA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912718718 (17h8m55.071484452s) [noise=true , address=77.90.31.18:4806 , piece_id=RRTHOAKZSK6ZKWLXCMODWX5HZ5I62OMUEBB64FS6RSDLMNL2OESA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912890155 (17h8m55.071655889s) [noise=true , address=45.91.116.4:4937 , piece_id=AZQ6DES5FGD63E56HNPDEOJJUZNKOF3ZYU33OMOUX47L7ZGYXISQ, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912973765 (17h8m55.071739499s) [noise=true , address=104.36.82.4:4802 , piece_id=6TH5NRIXI5GLRDDZDKZJZWPQYCK7BTMF2U5TDQYOV3SL227M6IZA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913095419 (17h8m55.071861153s) [noise=true , address=104.36.82.10:4079 , piece_id=RSWLIXFVVRXKXD6TI4TDKEPW3HY6XU5AUAAH5N3X4UIFNEU3H5CQ, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913202434 (17h8m55.071968168s) [noise=true , address=77.90.31.15:4697 , piece_id=VLJCK73RVDCWWPAC7HUVHLQV2AYUPRYNEA7SNSO7P2ZMRK2FOLSQ, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913320449 (17h8m55.072086183s) [noise=true , address=45.91.116.9:4263 , piece_id=RYI4SAUA6IXKXAAHC62LDOKQUKQNTEG7QCG4QHUGMSREVGWEMNGA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913452622 (17h8m55.072218356s) [noise=true , address=77.90.31.12:4592 , piece_id=4RLI6WE2HS6N2QRLJQZXSYG65EIMVROLJVYNW56CSY254ZLREUYA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913534449 (17h8m55.072300183s) [noise=true , address=104.36.82.8:4971 , piece_id=UZT2NT7JR4F7NIXPSY2ORFBQVJOXZSPEJCSLFAHPH4BDSLY2W3DA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913650701 (17h8m55.072416435s) [noise=true , address=77.90.31.12:4589 , piece_id=L6ZTZ5PJPV2WM2BZGGAJEKBS4B2TS5S55PSD7WGA2YRH3MTLG5GA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:29.900914740 (17h8m57.059680464s) [upload=14994]: Next returned nil segment Aug 26 09:12:29.900934336 (17h8m57.059700081s) [upload=14994]: Waiting for error group managing segments... Aug 26 09:12:30.221707715 (17h8m57.380473449s) [noise=true , address=77.90.31.18:4806 , piece_id=RRTHOAKZSK6ZKWLXCMODWX5HZ5I62OMUEBB64FS6RSDLMNL2OESA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.225253190 (17h8m57.384018924s) [noise=true , address=77.90.31.12:4592 , piece_id=4RLI6WE2HS6N2QRLJQZXSYG65EIMVROLJVYNW56CSY254ZLREUYA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.228153169 (17h8m57.386918893s) [noise=true , address=45.91.116.4:4937 , piece_id=AZQ6DES5FGD63E56HNPDEOJJUZNKOF3ZYU33OMOUX47L7ZGYXISQ, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.244451243 (17h8m57.403216977s) [noise=true , address=45.91.116.9:4263 , piece_id=RYI4SAUA6IXKXAAHC62LDOKQUKQNTEG7QCG4QHUGMSREVGWEMNGA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.274597415 (17h8m57.433363149s) [noise=true , address=45.91.116.5:4299 , piece_id=G2JPCAKEMRINSSYZ24B362KA54Q2T6HDEGJQFHVWNEJGGFGRR2KA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.280700783 (17h8m57.439466517s) [noise=true , address=104.36.82.4:4802 , piece_id=6TH5NRIXI5GLRDDZDKZJZWPQYCK7BTMF2U5TDQYOV3SL227M6IZA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.294073090 (17h8m57.452838824s) [noise=true , address=104.36.82.7:4912 , piece_id=OYVJF5VJJMZGE6E2JREPMGIZGWKQQWAHK4WCUHXFATZZGJXFJW2Q, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.310065069 (17h8m57.468830803s) [noise=true , address=104.36.82.8:4971 , piece_id=UZT2NT7JR4F7NIXPSY2ORFBQVJOXZSPEJCSLFAHPH4BDSLY2W3DA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.310725412 (17h8m57.469491146s) [noise=true , address=45.91.116.10:4081 , piece_id=YYCAJYYQ4MXTVLBHWA6R66TIAHAEJOUQ7BBH6A3PVB56MCXUXCDA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.327312570 (17h8m57.486078304s) [noise=true , address=77.90.31.15:4697 , piece_id=VLJCK73RVDCWWPAC7HUVHLQV2AYUPRYNEA7SNSO7P2ZMRK2FOLSQ, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.371703187 (17h8m57.530468921s) [noise=true , address=77.90.31.16:4735 , piece_id=F5ZXRAIRT3GGYVDCOHQWEAWDTJ5DN2OF2AW5YB3W3MTHTUUTVJ6Q, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.427020261 (17h8m57.585785996s) [noise=true , address=45.91.116.11:4119 , piece_id=RRRPLNZBTVHIQTUMONDVTIBCMEBLBM2RPFCOHX7RBX4HFGZUYS3Q, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.436406982 (17h8m57.595172716s) [noise=true , address=45.91.116.14:4191 , piece_id=YO65KQXJOPP2BVDRQTEURGV2RDY3M36W4JSPPBTM6LKVHZ7UIEXA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.436435246 (17h8m57.59520098s) [seg_pos={0 0}, upload=14994]: Segment reached optimal threshold of 13 pieces. Aug 26 09:12:30.436615631 (17h8m57.595381365s) [noise=true , address=77.90.31.12:4589 , piece_id=L6ZTZ5PJPV2WM2BZGGAJEKBS4B2TS5S55PSD7WGA2YRH3MTLG5GA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Aug 26 09:12:30.436693600 (17h8m57.595459334s) [noise=true , address=45.91.116.9:4262 , piece_id=WRDGDP5TBAYZTRPBYDW4ZESCR7NJJHVBU6WP4UYC4B3V5D7ICOMA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: write tcp 188.166.225.170:55590->45.91.116.9:4262: use of closed network connection; write tcp 188.166.225.170:55590->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55590->45.91.116.9:4262: use of closed network connection Aug 26 09:12:30.436700754 (17h8m57.595466488s) [noise=true , address=104.36.82.10:4079 , piece_id=RSWLIXFVVRXKXD6TI4TDKEPW3HY6XU5AUAAH5N3X4UIFNEU3H5CQ, seg_pos={0 0}, upload=14994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Aug 26 09:12:30.436750138 (17h8m57.595515872s) [seg_pos={0 0}, upload=14994]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 09:12:30.436849909 (17h8m57.595615643s) [upload=14994]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 09:12:30.436855740 (17h8m57.595621474s) [upload=14994]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 09:12:30.739827348 (17h8m57.898593082s) [upload=14994]: Done uploading segments... Aug 26 09:12:30.739846063 (17h8m57.898611797s) [upload=14994]: Upload finished. err: <nil> Duration: 8.414443431s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 4.944541426s FAIL (2025-08-26 09:14:42 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 4.944612042s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................X...X..] duration: 13.197542152s FAIL (2025-08-26 09:14:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 09:12:27.460801938 (17h8m54.619567672s) [upload=14994]: Starting upload Aug 26 09:12:27.460861261 (17h8m54.619626995s) [upload=14994]: Done starting upload Aug 26 09:12:27.460967244 (17h8m54.619732978s) [upload=14994]: Uploading segments... Aug 26 09:12:27.461061044 (17h8m54.619826768s) [upload=14994]: Got next segment. Inline: false Aug 26 09:12:27.461137420 (17h8m54.619903154s) [upload=14994]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 09:12:27.461142410 (17h8m54.619908144s) [upload=14994]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 09:12:27.911433899 (17h8m55.070199633s) [seg_pos={0 0}, upload=14994]: Begin upload segment... Aug 26 09:12:27.911481470 (17h8m55.070247204s) [seg_pos={0 0}, upload=14994]: Done begin upload segment. Aug 26 09:12:27.911501920 (17h8m55.070267654s) [noise=true , address=104.36.82.7:4912 , piece_id=OYVJF5VJJMZGE6E2JREPMGIZGWKQQWAHK4WCUHXFATZZGJXFJW2Q, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.911554130 (17h8m55.070319904s) [noise=true , address=45.91.116.14:4191 , piece_id=YO65KQXJOPP2BVDRQTEURGV2RDY3M36W4JSPPBTM6LKVHZ7UIEXA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.911868270 (17h8m55.070634004s) [noise=true , address=45.91.116.11:4119 , piece_id=RRRPLNZBTVHIQTUMONDVTIBCMEBLBM2RPFCOHX7RBX4HFGZUYS3Q, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912181570 (17h8m55.070947305s) [noise=true , address=45.91.116.10:4081 , piece_id=YYCAJYYQ4MXTVLBHWA6R66TIAHAEJOUQ7BBH6A3PVB56MCXUXCDA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912244641 (17h8m55.071010365s) [noise=true , address=77.90.31.16:4735 , piece_id=F5ZXRAIRT3GGYVDCOHQWEAWDTJ5DN2OF2AW5YB3W3MTHTUUTVJ6Q, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912475152 (17h8m55.071240886s) [noise=true , address=45.91.116.5:4299 , piece_id=G2JPCAKEMRINSSYZ24B362KA54Q2T6HDEGJQFHVWNEJGGFGRR2KA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912684673 (17h8m55.071450407s) [noise=true , address=45.91.116.9:4262 , piece_id=WRDGDP5TBAYZTRPBYDW4ZESCR7NJJHVBU6WP4UYC4B3V5D7ICOMA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912718718 (17h8m55.071484452s) [noise=true , address=77.90.31.18:4806 , piece_id=RRTHOAKZSK6ZKWLXCMODWX5HZ5I62OMUEBB64FS6RSDLMNL2OESA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912890155 (17h8m55.071655889s) [noise=true , address=45.91.116.4:4937 , piece_id=AZQ6DES5FGD63E56HNPDEOJJUZNKOF3ZYU33OMOUX47L7ZGYXISQ, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912973765 (17h8m55.071739499s) [noise=true , address=104.36.82.4:4802 , piece_id=6TH5NRIXI5GLRDDZDKZJZWPQYCK7BTMF2U5TDQYOV3SL227M6IZA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913095419 (17h8m55.071861153s) [noise=true , address=104.36.82.10:4079 , piece_id=RSWLIXFVVRXKXD6TI4TDKEPW3HY6XU5AUAAH5N3X4UIFNEU3H5CQ, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913202434 (17h8m55.071968168s) [noise=true , address=77.90.31.15:4697 , piece_id=VLJCK73RVDCWWPAC7HUVHLQV2AYUPRYNEA7SNSO7P2ZMRK2FOLSQ, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913320449 (17h8m55.072086183s) [noise=true , address=45.91.116.9:4263 , piece_id=RYI4SAUA6IXKXAAHC62LDOKQUKQNTEG7QCG4QHUGMSREVGWEMNGA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913452622 (17h8m55.072218356s) [noise=true , address=77.90.31.12:4592 , piece_id=4RLI6WE2HS6N2QRLJQZXSYG65EIMVROLJVYNW56CSY254ZLREUYA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913534449 (17h8m55.072300183s) [noise=true , address=104.36.82.8:4971 , piece_id=UZT2NT7JR4F7NIXPSY2ORFBQVJOXZSPEJCSLFAHPH4BDSLY2W3DA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913650701 (17h8m55.072416435s) [noise=true , address=77.90.31.12:4589 , piece_id=L6ZTZ5PJPV2WM2BZGGAJEKBS4B2TS5S55PSD7WGA2YRH3MTLG5GA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:29.900914740 (17h8m57.059680464s) [upload=14994]: Next returned nil segment Aug 26 09:12:29.900934336 (17h8m57.059700081s) [upload=14994]: Waiting for error group managing segments... Aug 26 09:12:30.221707715 (17h8m57.380473449s) [noise=true , address=77.90.31.18:4806 , piece_id=RRTHOAKZSK6ZKWLXCMODWX5HZ5I62OMUEBB64FS6RSDLMNL2OESA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.225253190 (17h8m57.384018924s) [noise=true , address=77.90.31.12:4592 , piece_id=4RLI6WE2HS6N2QRLJQZXSYG65EIMVROLJVYNW56CSY254ZLREUYA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.228153169 (17h8m57.386918893s) [noise=true , address=45.91.116.4:4937 , piece_id=AZQ6DES5FGD63E56HNPDEOJJUZNKOF3ZYU33OMOUX47L7ZGYXISQ, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.244451243 (17h8m57.403216977s) [noise=true , address=45.91.116.9:4263 , piece_id=RYI4SAUA6IXKXAAHC62LDOKQUKQNTEG7QCG4QHUGMSREVGWEMNGA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.274597415 (17h8m57.433363149s) [noise=true , address=45.91.116.5:4299 , piece_id=G2JPCAKEMRINSSYZ24B362KA54Q2T6HDEGJQFHVWNEJGGFGRR2KA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.280700783 (17h8m57.439466517s) [noise=true , address=104.36.82.4:4802 , piece_id=6TH5NRIXI5GLRDDZDKZJZWPQYCK7BTMF2U5TDQYOV3SL227M6IZA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.294073090 (17h8m57.452838824s) [noise=true , address=104.36.82.7:4912 , piece_id=OYVJF5VJJMZGE6E2JREPMGIZGWKQQWAHK4WCUHXFATZZGJXFJW2Q, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.310065069 (17h8m57.468830803s) [noise=true , address=104.36.82.8:4971 , piece_id=UZT2NT7JR4F7NIXPSY2ORFBQVJOXZSPEJCSLFAHPH4BDSLY2W3DA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.310725412 (17h8m57.469491146s) [noise=true , address=45.91.116.10:4081 , piece_id=YYCAJYYQ4MXTVLBHWA6R66TIAHAEJOUQ7BBH6A3PVB56MCXUXCDA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.327312570 (17h8m57.486078304s) [noise=true , address=77.90.31.15:4697 , piece_id=VLJCK73RVDCWWPAC7HUVHLQV2AYUPRYNEA7SNSO7P2ZMRK2FOLSQ, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.371703187 (17h8m57.530468921s) [noise=true , address=77.90.31.16:4735 , piece_id=F5ZXRAIRT3GGYVDCOHQWEAWDTJ5DN2OF2AW5YB3W3MTHTUUTVJ6Q, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.427020261 (17h8m57.585785996s) [noise=true , address=45.91.116.11:4119 , piece_id=RRRPLNZBTVHIQTUMONDVTIBCMEBLBM2RPFCOHX7RBX4HFGZUYS3Q, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.436406982 (17h8m57.595172716s) [noise=true , address=45.91.116.14:4191 , piece_id=YO65KQXJOPP2BVDRQTEURGV2RDY3M36W4JSPPBTM6LKVHZ7UIEXA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.436435246 (17h8m57.59520098s) [seg_pos={0 0}, upload=14994]: Segment reached optimal threshold of 13 pieces. Aug 26 09:12:30.436615631 (17h8m57.595381365s) [noise=true , address=77.90.31.12:4589 , piece_id=L6ZTZ5PJPV2WM2BZGGAJEKBS4B2TS5S55PSD7WGA2YRH3MTLG5GA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Aug 26 09:12:30.436693600 (17h8m57.595459334s) [noise=true , address=45.91.116.9:4262 , piece_id=WRDGDP5TBAYZTRPBYDW4ZESCR7NJJHVBU6WP4UYC4B3V5D7ICOMA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: write tcp 188.166.225.170:55590->45.91.116.9:4262: use of closed network connection; write tcp 188.166.225.170:55590->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55590->45.91.116.9:4262: use of closed network connection Aug 26 09:12:30.436700754 (17h8m57.595466488s) [noise=true , address=104.36.82.10:4079 , piece_id=RSWLIXFVVRXKXD6TI4TDKEPW3HY6XU5AUAAH5N3X4UIFNEU3H5CQ, seg_pos={0 0}, upload=14994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Aug 26 09:12:30.436750138 (17h8m57.595515872s) [seg_pos={0 0}, upload=14994]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 09:12:30.436849909 (17h8m57.595615643s) [upload=14994]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 09:12:30.436855740 (17h8m57.595621474s) [upload=14994]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 09:12:30.739827348 (17h8m57.898593082s) [upload=14994]: Done uploading segments... Aug 26 09:12:30.739846063 (17h8m57.898611797s) [upload=14994]: Upload finished. err: <nil> Duration: 13.197569985s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................................X...X.] duration: 6.491044072s FAIL (2025-08-26 09:13:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 09:12:27.460801938 (17h8m54.619567672s) [upload=14994]: Starting upload Aug 26 09:12:27.460861261 (17h8m54.619626995s) [upload=14994]: Done starting upload Aug 26 09:12:27.460967244 (17h8m54.619732978s) [upload=14994]: Uploading segments... Aug 26 09:12:27.461061044 (17h8m54.619826768s) [upload=14994]: Got next segment. Inline: false Aug 26 09:12:27.461137420 (17h8m54.619903154s) [upload=14994]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 09:12:27.461142410 (17h8m54.619908144s) [upload=14994]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 09:12:27.911433899 (17h8m55.070199633s) [seg_pos={0 0}, upload=14994]: Begin upload segment... Aug 26 09:12:27.911481470 (17h8m55.070247204s) [seg_pos={0 0}, upload=14994]: Done begin upload segment. Aug 26 09:12:27.911501920 (17h8m55.070267654s) [noise=true , address=104.36.82.7:4912 , piece_id=OYVJF5VJJMZGE6E2JREPMGIZGWKQQWAHK4WCUHXFATZZGJXFJW2Q, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.911554130 (17h8m55.070319904s) [noise=true , address=45.91.116.14:4191 , piece_id=YO65KQXJOPP2BVDRQTEURGV2RDY3M36W4JSPPBTM6LKVHZ7UIEXA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.911868270 (17h8m55.070634004s) [noise=true , address=45.91.116.11:4119 , piece_id=RRRPLNZBTVHIQTUMONDVTIBCMEBLBM2RPFCOHX7RBX4HFGZUYS3Q, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912181570 (17h8m55.070947305s) [noise=true , address=45.91.116.10:4081 , piece_id=YYCAJYYQ4MXTVLBHWA6R66TIAHAEJOUQ7BBH6A3PVB56MCXUXCDA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912244641 (17h8m55.071010365s) [noise=true , address=77.90.31.16:4735 , piece_id=F5ZXRAIRT3GGYVDCOHQWEAWDTJ5DN2OF2AW5YB3W3MTHTUUTVJ6Q, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912475152 (17h8m55.071240886s) [noise=true , address=45.91.116.5:4299 , piece_id=G2JPCAKEMRINSSYZ24B362KA54Q2T6HDEGJQFHVWNEJGGFGRR2KA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912684673 (17h8m55.071450407s) [noise=true , address=45.91.116.9:4262 , piece_id=WRDGDP5TBAYZTRPBYDW4ZESCR7NJJHVBU6WP4UYC4B3V5D7ICOMA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912718718 (17h8m55.071484452s) [noise=true , address=77.90.31.18:4806 , piece_id=RRTHOAKZSK6ZKWLXCMODWX5HZ5I62OMUEBB64FS6RSDLMNL2OESA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912890155 (17h8m55.071655889s) [noise=true , address=45.91.116.4:4937 , piece_id=AZQ6DES5FGD63E56HNPDEOJJUZNKOF3ZYU33OMOUX47L7ZGYXISQ, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912973765 (17h8m55.071739499s) [noise=true , address=104.36.82.4:4802 , piece_id=6TH5NRIXI5GLRDDZDKZJZWPQYCK7BTMF2U5TDQYOV3SL227M6IZA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913095419 (17h8m55.071861153s) [noise=true , address=104.36.82.10:4079 , piece_id=RSWLIXFVVRXKXD6TI4TDKEPW3HY6XU5AUAAH5N3X4UIFNEU3H5CQ, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913202434 (17h8m55.071968168s) [noise=true , address=77.90.31.15:4697 , piece_id=VLJCK73RVDCWWPAC7HUVHLQV2AYUPRYNEA7SNSO7P2ZMRK2FOLSQ, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913320449 (17h8m55.072086183s) [noise=true , address=45.91.116.9:4263 , piece_id=RYI4SAUA6IXKXAAHC62LDOKQUKQNTEG7QCG4QHUGMSREVGWEMNGA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913452622 (17h8m55.072218356s) [noise=true , address=77.90.31.12:4592 , piece_id=4RLI6WE2HS6N2QRLJQZXSYG65EIMVROLJVYNW56CSY254ZLREUYA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913534449 (17h8m55.072300183s) [noise=true , address=104.36.82.8:4971 , piece_id=UZT2NT7JR4F7NIXPSY2ORFBQVJOXZSPEJCSLFAHPH4BDSLY2W3DA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913650701 (17h8m55.072416435s) [noise=true , address=77.90.31.12:4589 , piece_id=L6ZTZ5PJPV2WM2BZGGAJEKBS4B2TS5S55PSD7WGA2YRH3MTLG5GA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:29.900914740 (17h8m57.059680464s) [upload=14994]: Next returned nil segment Aug 26 09:12:29.900934336 (17h8m57.059700081s) [upload=14994]: Waiting for error group managing segments... Aug 26 09:12:30.221707715 (17h8m57.380473449s) [noise=true , address=77.90.31.18:4806 , piece_id=RRTHOAKZSK6ZKWLXCMODWX5HZ5I62OMUEBB64FS6RSDLMNL2OESA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.225253190 (17h8m57.384018924s) [noise=true , address=77.90.31.12:4592 , piece_id=4RLI6WE2HS6N2QRLJQZXSYG65EIMVROLJVYNW56CSY254ZLREUYA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.228153169 (17h8m57.386918893s) [noise=true , address=45.91.116.4:4937 , piece_id=AZQ6DES5FGD63E56HNPDEOJJUZNKOF3ZYU33OMOUX47L7ZGYXISQ, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.244451243 (17h8m57.403216977s) [noise=true , address=45.91.116.9:4263 , piece_id=RYI4SAUA6IXKXAAHC62LDOKQUKQNTEG7QCG4QHUGMSREVGWEMNGA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.274597415 (17h8m57.433363149s) [noise=true , address=45.91.116.5:4299 , piece_id=G2JPCAKEMRINSSYZ24B362KA54Q2T6HDEGJQFHVWNEJGGFGRR2KA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.280700783 (17h8m57.439466517s) [noise=true , address=104.36.82.4:4802 , piece_id=6TH5NRIXI5GLRDDZDKZJZWPQYCK7BTMF2U5TDQYOV3SL227M6IZA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.294073090 (17h8m57.452838824s) [noise=true , address=104.36.82.7:4912 , piece_id=OYVJF5VJJMZGE6E2JREPMGIZGWKQQWAHK4WCUHXFATZZGJXFJW2Q, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.310065069 (17h8m57.468830803s) [noise=true , address=104.36.82.8:4971 , piece_id=UZT2NT7JR4F7NIXPSY2ORFBQVJOXZSPEJCSLFAHPH4BDSLY2W3DA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.310725412 (17h8m57.469491146s) [noise=true , address=45.91.116.10:4081 , piece_id=YYCAJYYQ4MXTVLBHWA6R66TIAHAEJOUQ7BBH6A3PVB56MCXUXCDA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.327312570 (17h8m57.486078304s) [noise=true , address=77.90.31.15:4697 , piece_id=VLJCK73RVDCWWPAC7HUVHLQV2AYUPRYNEA7SNSO7P2ZMRK2FOLSQ, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.371703187 (17h8m57.530468921s) [noise=true , address=77.90.31.16:4735 , piece_id=F5ZXRAIRT3GGYVDCOHQWEAWDTJ5DN2OF2AW5YB3W3MTHTUUTVJ6Q, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.427020261 (17h8m57.585785996s) [noise=true , address=45.91.116.11:4119 , piece_id=RRRPLNZBTVHIQTUMONDVTIBCMEBLBM2RPFCOHX7RBX4HFGZUYS3Q, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.436406982 (17h8m57.595172716s) [noise=true , address=45.91.116.14:4191 , piece_id=YO65KQXJOPP2BVDRQTEURGV2RDY3M36W4JSPPBTM6LKVHZ7UIEXA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.436435246 (17h8m57.59520098s) [seg_pos={0 0}, upload=14994]: Segment reached optimal threshold of 13 pieces. Aug 26 09:12:30.436615631 (17h8m57.595381365s) [noise=true , address=77.90.31.12:4589 , piece_id=L6ZTZ5PJPV2WM2BZGGAJEKBS4B2TS5S55PSD7WGA2YRH3MTLG5GA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Aug 26 09:12:30.436693600 (17h8m57.595459334s) [noise=true , address=45.91.116.9:4262 , piece_id=WRDGDP5TBAYZTRPBYDW4ZESCR7NJJHVBU6WP4UYC4B3V5D7ICOMA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: write tcp 188.166.225.170:55590->45.91.116.9:4262: use of closed network connection; write tcp 188.166.225.170:55590->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55590->45.91.116.9:4262: use of closed network connection Aug 26 09:12:30.436700754 (17h8m57.595466488s) [noise=true , address=104.36.82.10:4079 , piece_id=RSWLIXFVVRXKXD6TI4TDKEPW3HY6XU5AUAAH5N3X4UIFNEU3H5CQ, seg_pos={0 0}, upload=14994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Aug 26 09:12:30.436750138 (17h8m57.595515872s) [seg_pos={0 0}, upload=14994]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 09:12:30.436849909 (17h8m57.595615643s) [upload=14994]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 09:12:30.436855740 (17h8m57.595621474s) [upload=14994]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 09:12:30.739827348 (17h8m57.898593082s) [upload=14994]: Done uploading segments... Aug 26 09:12:30.739846063 (17h8m57.898611797s) [upload=14994]: Upload finished. err: <nil> Duration: 6.491072937s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 4.999869682s FAIL (2025-08-26 09:13:30 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 4.999901974s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 4.808701888s FAIL (2025-08-26 09:13:05 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 4.808730975s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................................X...X] duration: 30.001981812s FAIL (2025-08-26 09:12:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 09:12:27.460801938 (17h8m54.619567672s) [upload=14994]: Starting upload Aug 26 09:12:27.460861261 (17h8m54.619626995s) [upload=14994]: Done starting upload Aug 26 09:12:27.460967244 (17h8m54.619732978s) [upload=14994]: Uploading segments... Aug 26 09:12:27.461061044 (17h8m54.619826768s) [upload=14994]: Got next segment. Inline: false Aug 26 09:12:27.461137420 (17h8m54.619903154s) [upload=14994]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 09:12:27.461142410 (17h8m54.619908144s) [upload=14994]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 09:12:27.911433899 (17h8m55.070199633s) [seg_pos={0 0}, upload=14994]: Begin upload segment... Aug 26 09:12:27.911481470 (17h8m55.070247204s) [seg_pos={0 0}, upload=14994]: Done begin upload segment. Aug 26 09:12:27.911501920 (17h8m55.070267654s) [noise=true , address=104.36.82.7:4912 , piece_id=OYVJF5VJJMZGE6E2JREPMGIZGWKQQWAHK4WCUHXFATZZGJXFJW2Q, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.911554130 (17h8m55.070319904s) [noise=true , address=45.91.116.14:4191 , piece_id=YO65KQXJOPP2BVDRQTEURGV2RDY3M36W4JSPPBTM6LKVHZ7UIEXA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.911868270 (17h8m55.070634004s) [noise=true , address=45.91.116.11:4119 , piece_id=RRRPLNZBTVHIQTUMONDVTIBCMEBLBM2RPFCOHX7RBX4HFGZUYS3Q, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912181570 (17h8m55.070947305s) [noise=true , address=45.91.116.10:4081 , piece_id=YYCAJYYQ4MXTVLBHWA6R66TIAHAEJOUQ7BBH6A3PVB56MCXUXCDA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912244641 (17h8m55.071010365s) [noise=true , address=77.90.31.16:4735 , piece_id=F5ZXRAIRT3GGYVDCOHQWEAWDTJ5DN2OF2AW5YB3W3MTHTUUTVJ6Q, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912475152 (17h8m55.071240886s) [noise=true , address=45.91.116.5:4299 , piece_id=G2JPCAKEMRINSSYZ24B362KA54Q2T6HDEGJQFHVWNEJGGFGRR2KA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912684673 (17h8m55.071450407s) [noise=true , address=45.91.116.9:4262 , piece_id=WRDGDP5TBAYZTRPBYDW4ZESCR7NJJHVBU6WP4UYC4B3V5D7ICOMA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912718718 (17h8m55.071484452s) [noise=true , address=77.90.31.18:4806 , piece_id=RRTHOAKZSK6ZKWLXCMODWX5HZ5I62OMUEBB64FS6RSDLMNL2OESA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912890155 (17h8m55.071655889s) [noise=true , address=45.91.116.4:4937 , piece_id=AZQ6DES5FGD63E56HNPDEOJJUZNKOF3ZYU33OMOUX47L7ZGYXISQ, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.912973765 (17h8m55.071739499s) [noise=true , address=104.36.82.4:4802 , piece_id=6TH5NRIXI5GLRDDZDKZJZWPQYCK7BTMF2U5TDQYOV3SL227M6IZA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913095419 (17h8m55.071861153s) [noise=true , address=104.36.82.10:4079 , piece_id=RSWLIXFVVRXKXD6TI4TDKEPW3HY6XU5AUAAH5N3X4UIFNEU3H5CQ, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913202434 (17h8m55.071968168s) [noise=true , address=77.90.31.15:4697 , piece_id=VLJCK73RVDCWWPAC7HUVHLQV2AYUPRYNEA7SNSO7P2ZMRK2FOLSQ, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913320449 (17h8m55.072086183s) [noise=true , address=45.91.116.9:4263 , piece_id=RYI4SAUA6IXKXAAHC62LDOKQUKQNTEG7QCG4QHUGMSREVGWEMNGA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913452622 (17h8m55.072218356s) [noise=true , address=77.90.31.12:4592 , piece_id=4RLI6WE2HS6N2QRLJQZXSYG65EIMVROLJVYNW56CSY254ZLREUYA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913534449 (17h8m55.072300183s) [noise=true , address=104.36.82.8:4971 , piece_id=UZT2NT7JR4F7NIXPSY2ORFBQVJOXZSPEJCSLFAHPH4BDSLY2W3DA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:27.913650701 (17h8m55.072416435s) [noise=true , address=77.90.31.12:4589 , piece_id=L6ZTZ5PJPV2WM2BZGGAJEKBS4B2TS5S55PSD7WGA2YRH3MTLG5GA, seg_pos={0 0}, upload=14994]: Uploading piece... Aug 26 09:12:29.900914740 (17h8m57.059680464s) [upload=14994]: Next returned nil segment Aug 26 09:12:29.900934336 (17h8m57.059700081s) [upload=14994]: Waiting for error group managing segments... Aug 26 09:12:30.221707715 (17h8m57.380473449s) [noise=true , address=77.90.31.18:4806 , piece_id=RRTHOAKZSK6ZKWLXCMODWX5HZ5I62OMUEBB64FS6RSDLMNL2OESA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.225253190 (17h8m57.384018924s) [noise=true , address=77.90.31.12:4592 , piece_id=4RLI6WE2HS6N2QRLJQZXSYG65EIMVROLJVYNW56CSY254ZLREUYA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.228153169 (17h8m57.386918893s) [noise=true , address=45.91.116.4:4937 , piece_id=AZQ6DES5FGD63E56HNPDEOJJUZNKOF3ZYU33OMOUX47L7ZGYXISQ, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.244451243 (17h8m57.403216977s) [noise=true , address=45.91.116.9:4263 , piece_id=RYI4SAUA6IXKXAAHC62LDOKQUKQNTEG7QCG4QHUGMSREVGWEMNGA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.274597415 (17h8m57.433363149s) [noise=true , address=45.91.116.5:4299 , piece_id=G2JPCAKEMRINSSYZ24B362KA54Q2T6HDEGJQFHVWNEJGGFGRR2KA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.280700783 (17h8m57.439466517s) [noise=true , address=104.36.82.4:4802 , piece_id=6TH5NRIXI5GLRDDZDKZJZWPQYCK7BTMF2U5TDQYOV3SL227M6IZA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.294073090 (17h8m57.452838824s) [noise=true , address=104.36.82.7:4912 , piece_id=OYVJF5VJJMZGE6E2JREPMGIZGWKQQWAHK4WCUHXFATZZGJXFJW2Q, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.310065069 (17h8m57.468830803s) [noise=true , address=104.36.82.8:4971 , piece_id=UZT2NT7JR4F7NIXPSY2ORFBQVJOXZSPEJCSLFAHPH4BDSLY2W3DA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.310725412 (17h8m57.469491146s) [noise=true , address=45.91.116.10:4081 , piece_id=YYCAJYYQ4MXTVLBHWA6R66TIAHAEJOUQ7BBH6A3PVB56MCXUXCDA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.327312570 (17h8m57.486078304s) [noise=true , address=77.90.31.15:4697 , piece_id=VLJCK73RVDCWWPAC7HUVHLQV2AYUPRYNEA7SNSO7P2ZMRK2FOLSQ, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.371703187 (17h8m57.530468921s) [noise=true , address=77.90.31.16:4735 , piece_id=F5ZXRAIRT3GGYVDCOHQWEAWDTJ5DN2OF2AW5YB3W3MTHTUUTVJ6Q, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.427020261 (17h8m57.585785996s) [noise=true , address=45.91.116.11:4119 , piece_id=RRRPLNZBTVHIQTUMONDVTIBCMEBLBM2RPFCOHX7RBX4HFGZUYS3Q, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.436406982 (17h8m57.595172716s) [noise=true , address=45.91.116.14:4191 , piece_id=YO65KQXJOPP2BVDRQTEURGV2RDY3M36W4JSPPBTM6LKVHZ7UIEXA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: <nil> Aug 26 09:12:30.436435246 (17h8m57.59520098s) [seg_pos={0 0}, upload=14994]: Segment reached optimal threshold of 13 pieces. Aug 26 09:12:30.436615631 (17h8m57.595381365s) [noise=true , address=77.90.31.12:4589 , piece_id=L6ZTZ5PJPV2WM2BZGGAJEKBS4B2TS5S55PSD7WGA2YRH3MTLG5GA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Aug 26 09:12:30.436693600 (17h8m57.595459334s) [noise=true , address=45.91.116.9:4262 , piece_id=WRDGDP5TBAYZTRPBYDW4ZESCR7NJJHVBU6WP4UYC4B3V5D7ICOMA, seg_pos={0 0}, upload=14994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: write tcp 188.166.225.170:55590->45.91.116.9:4262: use of closed network connection; write tcp 188.166.225.170:55590->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55590->45.91.116.9:4262: use of closed network connection Aug 26 09:12:30.436700754 (17h8m57.595466488s) [noise=true , address=104.36.82.10:4079 , piece_id=RSWLIXFVVRXKXD6TI4TDKEPW3HY6XU5AUAAH5N3X4UIFNEU3H5CQ, seg_pos={0 0}, upload=14994]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Aug 26 09:12:30.436750138 (17h8m57.595515872s) [seg_pos={0 0}, upload=14994]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 09:12:30.436849909 (17h8m57.595615643s) [upload=14994]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 09:12:30.436855740 (17h8m57.595621474s) [upload=14994]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 09:12:30.739827348 (17h8m57.898593082s) [upload=14994]: Done uploading segments... Aug 26 09:12:30.739846063 (17h8m57.898611797s) [upload=14994]: Upload finished. err: <nil> Duration: 30.002023401s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.107375953s FAIL (2025-08-26 09:12:27 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 5.10742177s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 15.16543077s FAIL (2025-08-26 09:12:05 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 15.165451889s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.000828887s FAIL (2025-08-26 09:11:49 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.000848594s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 7.586651679s FAIL (2025-08-26 09:11:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 09:09:04.400846680 (17h5m31.559612414s) [upload=14949]: Starting upload Aug 26 09:09:04.400928166 (17h5m31.5596939s) [upload=14949]: Done starting upload Aug 26 09:09:04.401133950 (17h5m31.559899684s) [upload=14949]: Uploading segments... Aug 26 09:09:04.401176491 (17h5m31.559942225s) [upload=14949]: Got next segment. Inline: false Aug 26 09:09:04.401217931 (17h5m31.559983665s) [upload=14949]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 09:09:04.401222579 (17h5m31.559988313s) [upload=14949]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 09:09:04.635826582 (17h5m31.794592316s) [seg_pos={0 0}, upload=14949]: Begin upload segment... Aug 26 09:09:04.635872490 (17h5m31.794638224s) [seg_pos={0 0}, upload=14949]: Done begin upload segment. Aug 26 09:09:04.635900484 (17h5m31.794666218s) [noise=true , address=45.91.116.7:4009 , piece_id=L7EB5ZDLF2LUB736IFS4D634XUJ6EJXO6KJAPQ35NGDJBCQSRWZA, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.636001216 (17h5m31.79476694s) [noise=true , address=104.36.82.3:4767 , piece_id=HRZHMJIL3XJK62XWNCE3EFEWCBS73N4B4VYZCTKMTE7HNVEVWRTQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.636372827 (17h5m31.795138561s) [noise=true , address=104.36.82.7:4911 , piece_id=YFT74QCKM25TOXFVN7TS3F2D7ERXV5T3HQ6EKGUGCDPBC2Z7PFOQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.636490372 (17h5m31.795256106s) [noise=true , address=104.36.82.10:4079 , piece_id=LXCOFVTNOJGDRYBG7REE2EGD5ULWCRNULY5QF2WZY4GMGA7NWJBA, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.636667622 (17h5m31.795433356s) [noise=true , address=77.90.31.12:4590 , piece_id=VQMJ54DJMYAUWMRGDD3M5BWNECL3RS6H55QLH65A2O6U3AXS77JQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.636792580 (17h5m31.795558314s) [noise=true , address=45.91.116.8:4048 , piece_id=LBVXV4JKEKQL2BWK746X54TC2O6NTWZP642P3LGNFFTCBZ3E4NOQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.636958477 (17h5m31.795724212s) [noise=true , address=45.91.116.14:4190 , piece_id=LTJ3OMHQACLNHGMBT5AOGT7NSFLRJM43EKXCKLACHRMCSXGBDSGA, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637083898 (17h5m31.795849632s) [noise=true , address=45.91.116.13:4153 , piece_id=NNBZSHJ223ADK2ZACOE6CU6NIPF3Q3RN4OIMDRADQD4I3SKRYUOA, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637253763 (17h5m31.796019497s) [noise=true , address=104.36.82.16:4294 , piece_id=4YAQ6NHPM7SQAGF4NU3MCNC34VZIQMRC67LT56K52GFYIN5DJRHQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637379794 (17h5m31.796145528s) [noise=true , address=45.91.116.7:4011 , piece_id=BTIO3GQNL7QIZIJ2RXVZNLL26QPS4HOINCXU7647575ZTCVGSH2Q, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637520994 (17h5m31.796286728s) [noise=true , address=45.91.116.4:4940 , piece_id=EYZG7LGEDSKLGBHCWM6SUNVVIWOQ73CDFW63POEI225ZFVCGWM4Q, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637632307 (17h5m31.796398031s) [noise=true , address=45.91.116.5:4299 , piece_id=JHTJ5K4NGYXU26RUG5J7BZJSPYUCHB5AOZOWZGFXVLJTGOFREAGQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637859251 (17h5m31.796624985s) [noise=true , address=45.91.116.12:4225 , piece_id=JUA7ZSFCBJIVIHT4O3AOM3PA52NI4MJVYT3WLITGPKTYV5CB2AZQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637904378 (17h5m31.796670112s) [noise=true , address=104.36.82.14:4221 , piece_id=SVBQQBB6F3NC6RJVRD7TAFO7K7IHMUINQLPUHVMXT6L5JF3MJGGA, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.638110251 (17h5m31.796875985s) [noise=true , address=104.36.82.2:4731 , piece_id=EXR7RQYCBV4KTHMIYES5JTVGNETIWY2BNP4FZB224VXTUWFNMGGQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.638356723 (17h5m31.797122457s) [noise=true , address=77.90.31.19:4843 , piece_id=4EGGCZ4DVXREGY2ZLJRAUEFODNOWVQFNCPY745NLW5F2MBCMIGRQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:06.586899392 (17h5m33.745665126s) [upload=14949]: Next returned nil segment Aug 26 09:09:06.586913299 (17h5m33.745679033s) [upload=14949]: Waiting for error group managing segments... Aug 26 09:09:06.913746253 (17h5m34.072511987s) [noise=true , address=45.91.116.14:4190 , piece_id=LTJ3OMHQACLNHGMBT5AOGT7NSFLRJM43EKXCKLACHRMCSXGBDSGA, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:06.916239183 (17h5m34.075004927s) [noise=true , address=45.91.116.5:4299 , piece_id=JHTJ5K4NGYXU26RUG5J7BZJSPYUCHB5AOZOWZGFXVLJTGOFREAGQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:06.928170453 (17h5m34.086936177s) [noise=true , address=77.90.31.12:4590 , piece_id=VQMJ54DJMYAUWMRGDD3M5BWNECL3RS6H55QLH65A2O6U3AXS77JQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:06.941967592 (17h5m34.100733326s) [noise=true , address=104.36.82.16:4294 , piece_id=4YAQ6NHPM7SQAGF4NU3MCNC34VZIQMRC67LT56K52GFYIN5DJRHQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:06.958041898 (17h5m34.116807632s) [noise=true , address=45.91.116.7:4011 , piece_id=BTIO3GQNL7QIZIJ2RXVZNLL26QPS4HOINCXU7647575ZTCVGSH2Q, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:06.964517579 (17h5m34.123283313s) [noise=true , address=104.36.82.3:4767 , piece_id=HRZHMJIL3XJK62XWNCE3EFEWCBS73N4B4VYZCTKMTE7HNVEVWRTQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:06.980430427 (17h5m34.139196161s) [noise=true , address=45.91.116.4:4940 , piece_id=EYZG7LGEDSKLGBHCWM6SUNVVIWOQ73CDFW63POEI225ZFVCGWM4Q, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.059703899 (17h5m34.218469633s) [noise=true , address=45.91.116.12:4225 , piece_id=JUA7ZSFCBJIVIHT4O3AOM3PA52NI4MJVYT3WLITGPKTYV5CB2AZQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.075077855 (17h5m34.233843589s) [noise=true , address=45.91.116.13:4153 , piece_id=NNBZSHJ223ADK2ZACOE6CU6NIPF3Q3RN4OIMDRADQD4I3SKRYUOA, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.079786514 (17h5m34.238552238s) [noise=true , address=77.90.31.19:4843 , piece_id=4EGGCZ4DVXREGY2ZLJRAUEFODNOWVQFNCPY745NLW5F2MBCMIGRQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.093401005 (17h5m34.252166739s) [noise=true , address=104.36.82.7:4911 , piece_id=YFT74QCKM25TOXFVN7TS3F2D7ERXV5T3HQ6EKGUGCDPBC2Z7PFOQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.146894530 (17h5m34.305660254s) [noise=true , address=104.36.82.2:4731 , piece_id=EXR7RQYCBV4KTHMIYES5JTVGNETIWY2BNP4FZB224VXTUWFNMGGQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.169721968 (17h5m34.328487702s) [noise=true , address=104.36.82.14:4221 , piece_id=SVBQQBB6F3NC6RJVRD7TAFO7K7IHMUINQLPUHVMXT6L5JF3MJGGA, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.169752937 (17h5m34.328518671s) [seg_pos={0 0}, upload=14949]: Segment reached optimal threshold of 13 pieces. Aug 26 09:09:07.170036880 (17h5m34.328802614s) [noise=true , address=104.36.82.10:4079 , piece_id=LXCOFVTNOJGDRYBG7REE2EGD5ULWCRNULY5QF2WZY4GMGA7NWJBA, seg_pos={0 0}, upload=14949]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: write tcp 188.166.225.170:33258->104.36.82.10:4079: use of closed network connection; write tcp 188.166.225.170:33258->104.36.82.10:4079: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33258->104.36.82.10:4079: use of closed network connection Aug 26 09:09:07.170051759 (17h5m34.328817493s) [noise=true , address=45.91.116.8:4048 , piece_id=LBVXV4JKEKQL2BWK746X54TC2O6NTWZP642P3LGNFFTCBZ3E4NOQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Aug 26 09:09:07.170096504 (17h5m34.328862238s) [noise=true , address=45.91.116.7:4009 , piece_id=L7EB5ZDLF2LUB736IFS4D634XUJ6EJXO6KJAPQ35NGDJBCQSRWZA, seg_pos={0 0}, upload=14949]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:48582->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:48582->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48582->45.91.116.7:4009: use of closed network connection Aug 26 09:09:07.170147161 (17h5m34.328912895s) [seg_pos={0 0}, upload=14949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 09:09:07.170239799 (17h5m34.329005533s) [upload=14949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 09:09:07.170244838 (17h5m34.329010583s) [upload=14949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 09:09:07.457893246 (17h5m34.616659s) [upload=14949]: Done uploading segments... Aug 26 09:09:07.457911631 (17h5m34.616677365s) [upload=14949]: Upload finished. err: <nil> Duration: 7.586687957s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.741981364s FAIL (2025-08-26 09:10:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 09:09:04.400846680 (17h5m31.559612414s) [upload=14949]: Starting upload Aug 26 09:09:04.400928166 (17h5m31.5596939s) [upload=14949]: Done starting upload Aug 26 09:09:04.401133950 (17h5m31.559899684s) [upload=14949]: Uploading segments... Aug 26 09:09:04.401176491 (17h5m31.559942225s) [upload=14949]: Got next segment. Inline: false Aug 26 09:09:04.401217931 (17h5m31.559983665s) [upload=14949]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 09:09:04.401222579 (17h5m31.559988313s) [upload=14949]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 09:09:04.635826582 (17h5m31.794592316s) [seg_pos={0 0}, upload=14949]: Begin upload segment... Aug 26 09:09:04.635872490 (17h5m31.794638224s) [seg_pos={0 0}, upload=14949]: Done begin upload segment. Aug 26 09:09:04.635900484 (17h5m31.794666218s) [noise=true , address=45.91.116.7:4009 , piece_id=L7EB5ZDLF2LUB736IFS4D634XUJ6EJXO6KJAPQ35NGDJBCQSRWZA, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.636001216 (17h5m31.79476694s) [noise=true , address=104.36.82.3:4767 , piece_id=HRZHMJIL3XJK62XWNCE3EFEWCBS73N4B4VYZCTKMTE7HNVEVWRTQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.636372827 (17h5m31.795138561s) [noise=true , address=104.36.82.7:4911 , piece_id=YFT74QCKM25TOXFVN7TS3F2D7ERXV5T3HQ6EKGUGCDPBC2Z7PFOQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.636490372 (17h5m31.795256106s) [noise=true , address=104.36.82.10:4079 , piece_id=LXCOFVTNOJGDRYBG7REE2EGD5ULWCRNULY5QF2WZY4GMGA7NWJBA, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.636667622 (17h5m31.795433356s) [noise=true , address=77.90.31.12:4590 , piece_id=VQMJ54DJMYAUWMRGDD3M5BWNECL3RS6H55QLH65A2O6U3AXS77JQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.636792580 (17h5m31.795558314s) [noise=true , address=45.91.116.8:4048 , piece_id=LBVXV4JKEKQL2BWK746X54TC2O6NTWZP642P3LGNFFTCBZ3E4NOQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.636958477 (17h5m31.795724212s) [noise=true , address=45.91.116.14:4190 , piece_id=LTJ3OMHQACLNHGMBT5AOGT7NSFLRJM43EKXCKLACHRMCSXGBDSGA, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637083898 (17h5m31.795849632s) [noise=true , address=45.91.116.13:4153 , piece_id=NNBZSHJ223ADK2ZACOE6CU6NIPF3Q3RN4OIMDRADQD4I3SKRYUOA, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637253763 (17h5m31.796019497s) [noise=true , address=104.36.82.16:4294 , piece_id=4YAQ6NHPM7SQAGF4NU3MCNC34VZIQMRC67LT56K52GFYIN5DJRHQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637379794 (17h5m31.796145528s) [noise=true , address=45.91.116.7:4011 , piece_id=BTIO3GQNL7QIZIJ2RXVZNLL26QPS4HOINCXU7647575ZTCVGSH2Q, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637520994 (17h5m31.796286728s) [noise=true , address=45.91.116.4:4940 , piece_id=EYZG7LGEDSKLGBHCWM6SUNVVIWOQ73CDFW63POEI225ZFVCGWM4Q, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637632307 (17h5m31.796398031s) [noise=true , address=45.91.116.5:4299 , piece_id=JHTJ5K4NGYXU26RUG5J7BZJSPYUCHB5AOZOWZGFXVLJTGOFREAGQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637859251 (17h5m31.796624985s) [noise=true , address=45.91.116.12:4225 , piece_id=JUA7ZSFCBJIVIHT4O3AOM3PA52NI4MJVYT3WLITGPKTYV5CB2AZQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637904378 (17h5m31.796670112s) [noise=true , address=104.36.82.14:4221 , piece_id=SVBQQBB6F3NC6RJVRD7TAFO7K7IHMUINQLPUHVMXT6L5JF3MJGGA, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.638110251 (17h5m31.796875985s) [noise=true , address=104.36.82.2:4731 , piece_id=EXR7RQYCBV4KTHMIYES5JTVGNETIWY2BNP4FZB224VXTUWFNMGGQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.638356723 (17h5m31.797122457s) [noise=true , address=77.90.31.19:4843 , piece_id=4EGGCZ4DVXREGY2ZLJRAUEFODNOWVQFNCPY745NLW5F2MBCMIGRQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:06.586899392 (17h5m33.745665126s) [upload=14949]: Next returned nil segment Aug 26 09:09:06.586913299 (17h5m33.745679033s) [upload=14949]: Waiting for error group managing segments... Aug 26 09:09:06.913746253 (17h5m34.072511987s) [noise=true , address=45.91.116.14:4190 , piece_id=LTJ3OMHQACLNHGMBT5AOGT7NSFLRJM43EKXCKLACHRMCSXGBDSGA, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:06.916239183 (17h5m34.075004927s) [noise=true , address=45.91.116.5:4299 , piece_id=JHTJ5K4NGYXU26RUG5J7BZJSPYUCHB5AOZOWZGFXVLJTGOFREAGQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:06.928170453 (17h5m34.086936177s) [noise=true , address=77.90.31.12:4590 , piece_id=VQMJ54DJMYAUWMRGDD3M5BWNECL3RS6H55QLH65A2O6U3AXS77JQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:06.941967592 (17h5m34.100733326s) [noise=true , address=104.36.82.16:4294 , piece_id=4YAQ6NHPM7SQAGF4NU3MCNC34VZIQMRC67LT56K52GFYIN5DJRHQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:06.958041898 (17h5m34.116807632s) [noise=true , address=45.91.116.7:4011 , piece_id=BTIO3GQNL7QIZIJ2RXVZNLL26QPS4HOINCXU7647575ZTCVGSH2Q, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:06.964517579 (17h5m34.123283313s) [noise=true , address=104.36.82.3:4767 , piece_id=HRZHMJIL3XJK62XWNCE3EFEWCBS73N4B4VYZCTKMTE7HNVEVWRTQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:06.980430427 (17h5m34.139196161s) [noise=true , address=45.91.116.4:4940 , piece_id=EYZG7LGEDSKLGBHCWM6SUNVVIWOQ73CDFW63POEI225ZFVCGWM4Q, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.059703899 (17h5m34.218469633s) [noise=true , address=45.91.116.12:4225 , piece_id=JUA7ZSFCBJIVIHT4O3AOM3PA52NI4MJVYT3WLITGPKTYV5CB2AZQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.075077855 (17h5m34.233843589s) [noise=true , address=45.91.116.13:4153 , piece_id=NNBZSHJ223ADK2ZACOE6CU6NIPF3Q3RN4OIMDRADQD4I3SKRYUOA, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.079786514 (17h5m34.238552238s) [noise=true , address=77.90.31.19:4843 , piece_id=4EGGCZ4DVXREGY2ZLJRAUEFODNOWVQFNCPY745NLW5F2MBCMIGRQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.093401005 (17h5m34.252166739s) [noise=true , address=104.36.82.7:4911 , piece_id=YFT74QCKM25TOXFVN7TS3F2D7ERXV5T3HQ6EKGUGCDPBC2Z7PFOQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.146894530 (17h5m34.305660254s) [noise=true , address=104.36.82.2:4731 , piece_id=EXR7RQYCBV4KTHMIYES5JTVGNETIWY2BNP4FZB224VXTUWFNMGGQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.169721968 (17h5m34.328487702s) [noise=true , address=104.36.82.14:4221 , piece_id=SVBQQBB6F3NC6RJVRD7TAFO7K7IHMUINQLPUHVMXT6L5JF3MJGGA, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.169752937 (17h5m34.328518671s) [seg_pos={0 0}, upload=14949]: Segment reached optimal threshold of 13 pieces. Aug 26 09:09:07.170036880 (17h5m34.328802614s) [noise=true , address=104.36.82.10:4079 , piece_id=LXCOFVTNOJGDRYBG7REE2EGD5ULWCRNULY5QF2WZY4GMGA7NWJBA, seg_pos={0 0}, upload=14949]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: write tcp 188.166.225.170:33258->104.36.82.10:4079: use of closed network connection; write tcp 188.166.225.170:33258->104.36.82.10:4079: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33258->104.36.82.10:4079: use of closed network connection Aug 26 09:09:07.170051759 (17h5m34.328817493s) [noise=true , address=45.91.116.8:4048 , piece_id=LBVXV4JKEKQL2BWK746X54TC2O6NTWZP642P3LGNFFTCBZ3E4NOQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Aug 26 09:09:07.170096504 (17h5m34.328862238s) [noise=true , address=45.91.116.7:4009 , piece_id=L7EB5ZDLF2LUB736IFS4D634XUJ6EJXO6KJAPQ35NGDJBCQSRWZA, seg_pos={0 0}, upload=14949]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:48582->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:48582->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48582->45.91.116.7:4009: use of closed network connection Aug 26 09:09:07.170147161 (17h5m34.328912895s) [seg_pos={0 0}, upload=14949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 09:09:07.170239799 (17h5m34.329005533s) [upload=14949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 09:09:07.170244838 (17h5m34.329010583s) [upload=14949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 09:09:07.457893246 (17h5m34.616659s) [upload=14949]: Done uploading segments... Aug 26 09:09:07.457911631 (17h5m34.616677365s) [upload=14949]: Upload finished. err: <nil> Duration: 6.742013375s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 6.751171058s FAIL (2025-08-26 09:10:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 09:09:04.400846680 (17h5m31.559612414s) [upload=14949]: Starting upload Aug 26 09:09:04.400928166 (17h5m31.5596939s) [upload=14949]: Done starting upload Aug 26 09:09:04.401133950 (17h5m31.559899684s) [upload=14949]: Uploading segments... Aug 26 09:09:04.401176491 (17h5m31.559942225s) [upload=14949]: Got next segment. Inline: false Aug 26 09:09:04.401217931 (17h5m31.559983665s) [upload=14949]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 09:09:04.401222579 (17h5m31.559988313s) [upload=14949]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 09:09:04.635826582 (17h5m31.794592316s) [seg_pos={0 0}, upload=14949]: Begin upload segment... Aug 26 09:09:04.635872490 (17h5m31.794638224s) [seg_pos={0 0}, upload=14949]: Done begin upload segment. Aug 26 09:09:04.635900484 (17h5m31.794666218s) [noise=true , address=45.91.116.7:4009 , piece_id=L7EB5ZDLF2LUB736IFS4D634XUJ6EJXO6KJAPQ35NGDJBCQSRWZA, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.636001216 (17h5m31.79476694s) [noise=true , address=104.36.82.3:4767 , piece_id=HRZHMJIL3XJK62XWNCE3EFEWCBS73N4B4VYZCTKMTE7HNVEVWRTQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.636372827 (17h5m31.795138561s) [noise=true , address=104.36.82.7:4911 , piece_id=YFT74QCKM25TOXFVN7TS3F2D7ERXV5T3HQ6EKGUGCDPBC2Z7PFOQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.636490372 (17h5m31.795256106s) [noise=true , address=104.36.82.10:4079 , piece_id=LXCOFVTNOJGDRYBG7REE2EGD5ULWCRNULY5QF2WZY4GMGA7NWJBA, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.636667622 (17h5m31.795433356s) [noise=true , address=77.90.31.12:4590 , piece_id=VQMJ54DJMYAUWMRGDD3M5BWNECL3RS6H55QLH65A2O6U3AXS77JQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.636792580 (17h5m31.795558314s) [noise=true , address=45.91.116.8:4048 , piece_id=LBVXV4JKEKQL2BWK746X54TC2O6NTWZP642P3LGNFFTCBZ3E4NOQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.636958477 (17h5m31.795724212s) [noise=true , address=45.91.116.14:4190 , piece_id=LTJ3OMHQACLNHGMBT5AOGT7NSFLRJM43EKXCKLACHRMCSXGBDSGA, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637083898 (17h5m31.795849632s) [noise=true , address=45.91.116.13:4153 , piece_id=NNBZSHJ223ADK2ZACOE6CU6NIPF3Q3RN4OIMDRADQD4I3SKRYUOA, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637253763 (17h5m31.796019497s) [noise=true , address=104.36.82.16:4294 , piece_id=4YAQ6NHPM7SQAGF4NU3MCNC34VZIQMRC67LT56K52GFYIN5DJRHQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637379794 (17h5m31.796145528s) [noise=true , address=45.91.116.7:4011 , piece_id=BTIO3GQNL7QIZIJ2RXVZNLL26QPS4HOINCXU7647575ZTCVGSH2Q, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637520994 (17h5m31.796286728s) [noise=true , address=45.91.116.4:4940 , piece_id=EYZG7LGEDSKLGBHCWM6SUNVVIWOQ73CDFW63POEI225ZFVCGWM4Q, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637632307 (17h5m31.796398031s) [noise=true , address=45.91.116.5:4299 , piece_id=JHTJ5K4NGYXU26RUG5J7BZJSPYUCHB5AOZOWZGFXVLJTGOFREAGQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637859251 (17h5m31.796624985s) [noise=true , address=45.91.116.12:4225 , piece_id=JUA7ZSFCBJIVIHT4O3AOM3PA52NI4MJVYT3WLITGPKTYV5CB2AZQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637904378 (17h5m31.796670112s) [noise=true , address=104.36.82.14:4221 , piece_id=SVBQQBB6F3NC6RJVRD7TAFO7K7IHMUINQLPUHVMXT6L5JF3MJGGA, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.638110251 (17h5m31.796875985s) [noise=true , address=104.36.82.2:4731 , piece_id=EXR7RQYCBV4KTHMIYES5JTVGNETIWY2BNP4FZB224VXTUWFNMGGQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.638356723 (17h5m31.797122457s) [noise=true , address=77.90.31.19:4843 , piece_id=4EGGCZ4DVXREGY2ZLJRAUEFODNOWVQFNCPY745NLW5F2MBCMIGRQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:06.586899392 (17h5m33.745665126s) [upload=14949]: Next returned nil segment Aug 26 09:09:06.586913299 (17h5m33.745679033s) [upload=14949]: Waiting for error group managing segments... Aug 26 09:09:06.913746253 (17h5m34.072511987s) [noise=true , address=45.91.116.14:4190 , piece_id=LTJ3OMHQACLNHGMBT5AOGT7NSFLRJM43EKXCKLACHRMCSXGBDSGA, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:06.916239183 (17h5m34.075004927s) [noise=true , address=45.91.116.5:4299 , piece_id=JHTJ5K4NGYXU26RUG5J7BZJSPYUCHB5AOZOWZGFXVLJTGOFREAGQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:06.928170453 (17h5m34.086936177s) [noise=true , address=77.90.31.12:4590 , piece_id=VQMJ54DJMYAUWMRGDD3M5BWNECL3RS6H55QLH65A2O6U3AXS77JQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:06.941967592 (17h5m34.100733326s) [noise=true , address=104.36.82.16:4294 , piece_id=4YAQ6NHPM7SQAGF4NU3MCNC34VZIQMRC67LT56K52GFYIN5DJRHQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:06.958041898 (17h5m34.116807632s) [noise=true , address=45.91.116.7:4011 , piece_id=BTIO3GQNL7QIZIJ2RXVZNLL26QPS4HOINCXU7647575ZTCVGSH2Q, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:06.964517579 (17h5m34.123283313s) [noise=true , address=104.36.82.3:4767 , piece_id=HRZHMJIL3XJK62XWNCE3EFEWCBS73N4B4VYZCTKMTE7HNVEVWRTQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:06.980430427 (17h5m34.139196161s) [noise=true , address=45.91.116.4:4940 , piece_id=EYZG7LGEDSKLGBHCWM6SUNVVIWOQ73CDFW63POEI225ZFVCGWM4Q, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.059703899 (17h5m34.218469633s) [noise=true , address=45.91.116.12:4225 , piece_id=JUA7ZSFCBJIVIHT4O3AOM3PA52NI4MJVYT3WLITGPKTYV5CB2AZQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.075077855 (17h5m34.233843589s) [noise=true , address=45.91.116.13:4153 , piece_id=NNBZSHJ223ADK2ZACOE6CU6NIPF3Q3RN4OIMDRADQD4I3SKRYUOA, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.079786514 (17h5m34.238552238s) [noise=true , address=77.90.31.19:4843 , piece_id=4EGGCZ4DVXREGY2ZLJRAUEFODNOWVQFNCPY745NLW5F2MBCMIGRQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.093401005 (17h5m34.252166739s) [noise=true , address=104.36.82.7:4911 , piece_id=YFT74QCKM25TOXFVN7TS3F2D7ERXV5T3HQ6EKGUGCDPBC2Z7PFOQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.146894530 (17h5m34.305660254s) [noise=true , address=104.36.82.2:4731 , piece_id=EXR7RQYCBV4KTHMIYES5JTVGNETIWY2BNP4FZB224VXTUWFNMGGQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.169721968 (17h5m34.328487702s) [noise=true , address=104.36.82.14:4221 , piece_id=SVBQQBB6F3NC6RJVRD7TAFO7K7IHMUINQLPUHVMXT6L5JF3MJGGA, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.169752937 (17h5m34.328518671s) [seg_pos={0 0}, upload=14949]: Segment reached optimal threshold of 13 pieces. Aug 26 09:09:07.170036880 (17h5m34.328802614s) [noise=true , address=104.36.82.10:4079 , piece_id=LXCOFVTNOJGDRYBG7REE2EGD5ULWCRNULY5QF2WZY4GMGA7NWJBA, seg_pos={0 0}, upload=14949]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: write tcp 188.166.225.170:33258->104.36.82.10:4079: use of closed network connection; write tcp 188.166.225.170:33258->104.36.82.10:4079: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33258->104.36.82.10:4079: use of closed network connection Aug 26 09:09:07.170051759 (17h5m34.328817493s) [noise=true , address=45.91.116.8:4048 , piece_id=LBVXV4JKEKQL2BWK746X54TC2O6NTWZP642P3LGNFFTCBZ3E4NOQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Aug 26 09:09:07.170096504 (17h5m34.328862238s) [noise=true , address=45.91.116.7:4009 , piece_id=L7EB5ZDLF2LUB736IFS4D634XUJ6EJXO6KJAPQ35NGDJBCQSRWZA, seg_pos={0 0}, upload=14949]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:48582->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:48582->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48582->45.91.116.7:4009: use of closed network connection Aug 26 09:09:07.170147161 (17h5m34.328912895s) [seg_pos={0 0}, upload=14949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 09:09:07.170239799 (17h5m34.329005533s) [upload=14949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 09:09:07.170244838 (17h5m34.329010583s) [upload=14949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 09:09:07.457893246 (17h5m34.616659s) [upload=14949]: Done uploading segments... Aug 26 09:09:07.457911631 (17h5m34.616677365s) [upload=14949]: Upload finished. err: <nil> Duration: 6.751200896s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.002381388s FAIL (2025-08-26 09:09:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 09:09:04.400846680 (17h5m31.559612414s) [upload=14949]: Starting upload Aug 26 09:09:04.400928166 (17h5m31.5596939s) [upload=14949]: Done starting upload Aug 26 09:09:04.401133950 (17h5m31.559899684s) [upload=14949]: Uploading segments... Aug 26 09:09:04.401176491 (17h5m31.559942225s) [upload=14949]: Got next segment. Inline: false Aug 26 09:09:04.401217931 (17h5m31.559983665s) [upload=14949]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 09:09:04.401222579 (17h5m31.559988313s) [upload=14949]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 09:09:04.635826582 (17h5m31.794592316s) [seg_pos={0 0}, upload=14949]: Begin upload segment... Aug 26 09:09:04.635872490 (17h5m31.794638224s) [seg_pos={0 0}, upload=14949]: Done begin upload segment. Aug 26 09:09:04.635900484 (17h5m31.794666218s) [noise=true , address=45.91.116.7:4009 , piece_id=L7EB5ZDLF2LUB736IFS4D634XUJ6EJXO6KJAPQ35NGDJBCQSRWZA, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.636001216 (17h5m31.79476694s) [noise=true , address=104.36.82.3:4767 , piece_id=HRZHMJIL3XJK62XWNCE3EFEWCBS73N4B4VYZCTKMTE7HNVEVWRTQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.636372827 (17h5m31.795138561s) [noise=true , address=104.36.82.7:4911 , piece_id=YFT74QCKM25TOXFVN7TS3F2D7ERXV5T3HQ6EKGUGCDPBC2Z7PFOQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.636490372 (17h5m31.795256106s) [noise=true , address=104.36.82.10:4079 , piece_id=LXCOFVTNOJGDRYBG7REE2EGD5ULWCRNULY5QF2WZY4GMGA7NWJBA, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.636667622 (17h5m31.795433356s) [noise=true , address=77.90.31.12:4590 , piece_id=VQMJ54DJMYAUWMRGDD3M5BWNECL3RS6H55QLH65A2O6U3AXS77JQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.636792580 (17h5m31.795558314s) [noise=true , address=45.91.116.8:4048 , piece_id=LBVXV4JKEKQL2BWK746X54TC2O6NTWZP642P3LGNFFTCBZ3E4NOQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.636958477 (17h5m31.795724212s) [noise=true , address=45.91.116.14:4190 , piece_id=LTJ3OMHQACLNHGMBT5AOGT7NSFLRJM43EKXCKLACHRMCSXGBDSGA, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637083898 (17h5m31.795849632s) [noise=true , address=45.91.116.13:4153 , piece_id=NNBZSHJ223ADK2ZACOE6CU6NIPF3Q3RN4OIMDRADQD4I3SKRYUOA, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637253763 (17h5m31.796019497s) [noise=true , address=104.36.82.16:4294 , piece_id=4YAQ6NHPM7SQAGF4NU3MCNC34VZIQMRC67LT56K52GFYIN5DJRHQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637379794 (17h5m31.796145528s) [noise=true , address=45.91.116.7:4011 , piece_id=BTIO3GQNL7QIZIJ2RXVZNLL26QPS4HOINCXU7647575ZTCVGSH2Q, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637520994 (17h5m31.796286728s) [noise=true , address=45.91.116.4:4940 , piece_id=EYZG7LGEDSKLGBHCWM6SUNVVIWOQ73CDFW63POEI225ZFVCGWM4Q, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637632307 (17h5m31.796398031s) [noise=true , address=45.91.116.5:4299 , piece_id=JHTJ5K4NGYXU26RUG5J7BZJSPYUCHB5AOZOWZGFXVLJTGOFREAGQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637859251 (17h5m31.796624985s) [noise=true , address=45.91.116.12:4225 , piece_id=JUA7ZSFCBJIVIHT4O3AOM3PA52NI4MJVYT3WLITGPKTYV5CB2AZQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.637904378 (17h5m31.796670112s) [noise=true , address=104.36.82.14:4221 , piece_id=SVBQQBB6F3NC6RJVRD7TAFO7K7IHMUINQLPUHVMXT6L5JF3MJGGA, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.638110251 (17h5m31.796875985s) [noise=true , address=104.36.82.2:4731 , piece_id=EXR7RQYCBV4KTHMIYES5JTVGNETIWY2BNP4FZB224VXTUWFNMGGQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:04.638356723 (17h5m31.797122457s) [noise=true , address=77.90.31.19:4843 , piece_id=4EGGCZ4DVXREGY2ZLJRAUEFODNOWVQFNCPY745NLW5F2MBCMIGRQ, seg_pos={0 0}, upload=14949]: Uploading piece... Aug 26 09:09:06.586899392 (17h5m33.745665126s) [upload=14949]: Next returned nil segment Aug 26 09:09:06.586913299 (17h5m33.745679033s) [upload=14949]: Waiting for error group managing segments... Aug 26 09:09:06.913746253 (17h5m34.072511987s) [noise=true , address=45.91.116.14:4190 , piece_id=LTJ3OMHQACLNHGMBT5AOGT7NSFLRJM43EKXCKLACHRMCSXGBDSGA, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:06.916239183 (17h5m34.075004927s) [noise=true , address=45.91.116.5:4299 , piece_id=JHTJ5K4NGYXU26RUG5J7BZJSPYUCHB5AOZOWZGFXVLJTGOFREAGQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:06.928170453 (17h5m34.086936177s) [noise=true , address=77.90.31.12:4590 , piece_id=VQMJ54DJMYAUWMRGDD3M5BWNECL3RS6H55QLH65A2O6U3AXS77JQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:06.941967592 (17h5m34.100733326s) [noise=true , address=104.36.82.16:4294 , piece_id=4YAQ6NHPM7SQAGF4NU3MCNC34VZIQMRC67LT56K52GFYIN5DJRHQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:06.958041898 (17h5m34.116807632s) [noise=true , address=45.91.116.7:4011 , piece_id=BTIO3GQNL7QIZIJ2RXVZNLL26QPS4HOINCXU7647575ZTCVGSH2Q, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:06.964517579 (17h5m34.123283313s) [noise=true , address=104.36.82.3:4767 , piece_id=HRZHMJIL3XJK62XWNCE3EFEWCBS73N4B4VYZCTKMTE7HNVEVWRTQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:06.980430427 (17h5m34.139196161s) [noise=true , address=45.91.116.4:4940 , piece_id=EYZG7LGEDSKLGBHCWM6SUNVVIWOQ73CDFW63POEI225ZFVCGWM4Q, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.059703899 (17h5m34.218469633s) [noise=true , address=45.91.116.12:4225 , piece_id=JUA7ZSFCBJIVIHT4O3AOM3PA52NI4MJVYT3WLITGPKTYV5CB2AZQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.075077855 (17h5m34.233843589s) [noise=true , address=45.91.116.13:4153 , piece_id=NNBZSHJ223ADK2ZACOE6CU6NIPF3Q3RN4OIMDRADQD4I3SKRYUOA, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.079786514 (17h5m34.238552238s) [noise=true , address=77.90.31.19:4843 , piece_id=4EGGCZ4DVXREGY2ZLJRAUEFODNOWVQFNCPY745NLW5F2MBCMIGRQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.093401005 (17h5m34.252166739s) [noise=true , address=104.36.82.7:4911 , piece_id=YFT74QCKM25TOXFVN7TS3F2D7ERXV5T3HQ6EKGUGCDPBC2Z7PFOQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.146894530 (17h5m34.305660254s) [noise=true , address=104.36.82.2:4731 , piece_id=EXR7RQYCBV4KTHMIYES5JTVGNETIWY2BNP4FZB224VXTUWFNMGGQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.169721968 (17h5m34.328487702s) [noise=true , address=104.36.82.14:4221 , piece_id=SVBQQBB6F3NC6RJVRD7TAFO7K7IHMUINQLPUHVMXT6L5JF3MJGGA, seg_pos={0 0}, upload=14949]: Done uploading piece. err: <nil> Aug 26 09:09:07.169752937 (17h5m34.328518671s) [seg_pos={0 0}, upload=14949]: Segment reached optimal threshold of 13 pieces. Aug 26 09:09:07.170036880 (17h5m34.328802614s) [noise=true , address=104.36.82.10:4079 , piece_id=LXCOFVTNOJGDRYBG7REE2EGD5ULWCRNULY5QF2WZY4GMGA7NWJBA, seg_pos={0 0}, upload=14949]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: write tcp 188.166.225.170:33258->104.36.82.10:4079: use of closed network connection; write tcp 188.166.225.170:33258->104.36.82.10:4079: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33258->104.36.82.10:4079: use of closed network connection Aug 26 09:09:07.170051759 (17h5m34.328817493s) [noise=true , address=45.91.116.8:4048 , piece_id=LBVXV4JKEKQL2BWK746X54TC2O6NTWZP642P3LGNFFTCBZ3E4NOQ, seg_pos={0 0}, upload=14949]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Aug 26 09:09:07.170096504 (17h5m34.328862238s) [noise=true , address=45.91.116.7:4009 , piece_id=L7EB5ZDLF2LUB736IFS4D634XUJ6EJXO6KJAPQ35NGDJBCQSRWZA, seg_pos={0 0}, upload=14949]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:48582->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:48582->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48582->45.91.116.7:4009: use of closed network connection Aug 26 09:09:07.170147161 (17h5m34.328912895s) [seg_pos={0 0}, upload=14949]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 09:09:07.170239799 (17h5m34.329005533s) [upload=14949]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 09:09:07.170244838 (17h5m34.329010583s) [upload=14949]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 09:09:07.457893246 (17h5m34.616659s) [upload=14949]: Done uploading segments... Aug 26 09:09:07.457911631 (17h5m34.616677365s) [upload=14949]: Upload finished. err: <nil> Duration: 30.002405674s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.98200455s FAIL (2025-08-26 08:56:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 08:07:40.970349974 (16h4m8.129115698s) [upload=14051]: Starting upload Aug 26 08:07:40.970414868 (16h4m8.129180602s) [upload=14051]: Done starting upload Aug 26 08:07:40.970581266 (16h4m8.129347001s) [upload=14051]: Uploading segments... Aug 26 08:07:40.970610923 (16h4m8.129376647s) [upload=14051]: Got next segment. Inline: false Aug 26 08:07:40.970630972 (16h4m8.129396706s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 08:07:40.970634669 (16h4m8.129400403s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 08:07:41.325086070 (16h4m8.483851804s) [seg_pos={0 0}, upload=14051]: Begin upload segment... Aug 26 08:07:41.325126537 (16h4m8.483892271s) [seg_pos={0 0}, upload=14051]: Done begin upload segment. Aug 26 08:07:41.325149130 (16h4m8.483914854s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325190900 (16h4m8.483956634s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325532844 (16h4m8.484298578s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325708391 (16h4m8.484474115s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325913813 (16h4m8.484679547s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326099950 (16h4m8.484865684s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326210281 (16h4m8.484976015s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326446914 (16h4m8.485212648s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326495045 (16h4m8.485260779s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326778057 (16h4m8.485543791s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326803586 (16h4m8.48556932s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327065918 (16h4m8.485831652s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327156781 (16h4m8.485922515s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327280669 (16h4m8.486046403s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327439733 (16h4m8.486205467s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327490521 (16h4m8.486256255s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:43.233257592 (16h4m10.392023326s) [upload=14051]: Next returned nil segment Aug 26 08:07:43.233285976 (16h4m10.39205171s) [upload=14051]: Waiting for error group managing segments... Aug 26 08:07:43.691090233 (16h4m10.849855967s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.691200514 (16h4m10.849966248s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.704377618 (16h4m10.863143342s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.743908296 (16h4m10.90267403s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.751936938 (16h4m10.910702673s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.752355369 (16h4m10.911121103s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.791779473 (16h4m10.950545207s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.806507966 (16h4m10.96527369s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.809165941 (16h4m10.967931675s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.842162998 (16h4m11.000928732s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.848984690 (16h4m11.007750424s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.881281698 (16h4m11.040047432s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915580876 (16h4m11.07434661s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915616184 (16h4m11.074381918s) [seg_pos={0 0}, upload=14051]: Segment reached optimal threshold of 13 pieces. Aug 26 08:07:43.915943981 (16h4m11.074709705s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Aug 26 08:07:43.915976523 (16h4m11.074742368s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection Aug 26 08:07:43.916020267 (16h4m11.074786101s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection Aug 26 08:07:43.916083849 (16h4m11.074849583s) [seg_pos={0 0}, upload=14051]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 08:07:43.916183800 (16h4m11.074949544s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 08:07:43.916190372 (16h4m11.074956136s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 08:07:44.252445185 (16h4m11.411210919s) [upload=14051]: Done uploading segments... Aug 26 08:07:44.252466035 (16h4m11.411231769s) [upload=14051]: Upload finished. err: <nil> Duration: 5.982034589s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 5.893095432s FAIL (2025-08-26 08:55:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 08:07:40.970349974 (16h4m8.129115698s) [upload=14051]: Starting upload Aug 26 08:07:40.970414868 (16h4m8.129180602s) [upload=14051]: Done starting upload Aug 26 08:07:40.970581266 (16h4m8.129347001s) [upload=14051]: Uploading segments... Aug 26 08:07:40.970610923 (16h4m8.129376647s) [upload=14051]: Got next segment. Inline: false Aug 26 08:07:40.970630972 (16h4m8.129396706s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 08:07:40.970634669 (16h4m8.129400403s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 08:07:41.325086070 (16h4m8.483851804s) [seg_pos={0 0}, upload=14051]: Begin upload segment... Aug 26 08:07:41.325126537 (16h4m8.483892271s) [seg_pos={0 0}, upload=14051]: Done begin upload segment. Aug 26 08:07:41.325149130 (16h4m8.483914854s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325190900 (16h4m8.483956634s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325532844 (16h4m8.484298578s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325708391 (16h4m8.484474115s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325913813 (16h4m8.484679547s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326099950 (16h4m8.484865684s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326210281 (16h4m8.484976015s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326446914 (16h4m8.485212648s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326495045 (16h4m8.485260779s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326778057 (16h4m8.485543791s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326803586 (16h4m8.48556932s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327065918 (16h4m8.485831652s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327156781 (16h4m8.485922515s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327280669 (16h4m8.486046403s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327439733 (16h4m8.486205467s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327490521 (16h4m8.486256255s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:43.233257592 (16h4m10.392023326s) [upload=14051]: Next returned nil segment Aug 26 08:07:43.233285976 (16h4m10.39205171s) [upload=14051]: Waiting for error group managing segments... Aug 26 08:07:43.691090233 (16h4m10.849855967s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.691200514 (16h4m10.849966248s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.704377618 (16h4m10.863143342s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.743908296 (16h4m10.90267403s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.751936938 (16h4m10.910702673s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.752355369 (16h4m10.911121103s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.791779473 (16h4m10.950545207s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.806507966 (16h4m10.96527369s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.809165941 (16h4m10.967931675s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.842162998 (16h4m11.000928732s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.848984690 (16h4m11.007750424s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.881281698 (16h4m11.040047432s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915580876 (16h4m11.07434661s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915616184 (16h4m11.074381918s) [seg_pos={0 0}, upload=14051]: Segment reached optimal threshold of 13 pieces. Aug 26 08:07:43.915943981 (16h4m11.074709705s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Aug 26 08:07:43.915976523 (16h4m11.074742368s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection Aug 26 08:07:43.916020267 (16h4m11.074786101s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection Aug 26 08:07:43.916083849 (16h4m11.074849583s) [seg_pos={0 0}, upload=14051]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 08:07:43.916183800 (16h4m11.074949544s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 08:07:43.916190372 (16h4m11.074956136s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 08:07:44.252445185 (16h4m11.411210919s) [upload=14051]: Done uploading segments... Aug 26 08:07:44.252466035 (16h4m11.411231769s) [upload=14051]: Upload finished. err: <nil> Duration: 5.893121792s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [XXX.........................................................] duration: 5.820897647s FAIL (2025-08-26 08:54:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 08:07:40.970349974 (16h4m8.129115698s) [upload=14051]: Starting upload Aug 26 08:07:40.970414868 (16h4m8.129180602s) [upload=14051]: Done starting upload Aug 26 08:07:40.970581266 (16h4m8.129347001s) [upload=14051]: Uploading segments... Aug 26 08:07:40.970610923 (16h4m8.129376647s) [upload=14051]: Got next segment. Inline: false Aug 26 08:07:40.970630972 (16h4m8.129396706s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 08:07:40.970634669 (16h4m8.129400403s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 08:07:41.325086070 (16h4m8.483851804s) [seg_pos={0 0}, upload=14051]: Begin upload segment... Aug 26 08:07:41.325126537 (16h4m8.483892271s) [seg_pos={0 0}, upload=14051]: Done begin upload segment. Aug 26 08:07:41.325149130 (16h4m8.483914854s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325190900 (16h4m8.483956634s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325532844 (16h4m8.484298578s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325708391 (16h4m8.484474115s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325913813 (16h4m8.484679547s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326099950 (16h4m8.484865684s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326210281 (16h4m8.484976015s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326446914 (16h4m8.485212648s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326495045 (16h4m8.485260779s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326778057 (16h4m8.485543791s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326803586 (16h4m8.48556932s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327065918 (16h4m8.485831652s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327156781 (16h4m8.485922515s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327280669 (16h4m8.486046403s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327439733 (16h4m8.486205467s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327490521 (16h4m8.486256255s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:43.233257592 (16h4m10.392023326s) [upload=14051]: Next returned nil segment Aug 26 08:07:43.233285976 (16h4m10.39205171s) [upload=14051]: Waiting for error group managing segments... Aug 26 08:07:43.691090233 (16h4m10.849855967s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.691200514 (16h4m10.849966248s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.704377618 (16h4m10.863143342s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.743908296 (16h4m10.90267403s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.751936938 (16h4m10.910702673s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.752355369 (16h4m10.911121103s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.791779473 (16h4m10.950545207s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.806507966 (16h4m10.96527369s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.809165941 (16h4m10.967931675s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.842162998 (16h4m11.000928732s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.848984690 (16h4m11.007750424s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.881281698 (16h4m11.040047432s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915580876 (16h4m11.07434661s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915616184 (16h4m11.074381918s) [seg_pos={0 0}, upload=14051]: Segment reached optimal threshold of 13 pieces. Aug 26 08:07:43.915943981 (16h4m11.074709705s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Aug 26 08:07:43.915976523 (16h4m11.074742368s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection Aug 26 08:07:43.916020267 (16h4m11.074786101s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection Aug 26 08:07:43.916083849 (16h4m11.074849583s) [seg_pos={0 0}, upload=14051]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 08:07:43.916183800 (16h4m11.074949544s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 08:07:43.916190372 (16h4m11.074956136s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 08:07:44.252445185 (16h4m11.411210919s) [upload=14051]: Done uploading segments... Aug 26 08:07:44.252466035 (16h4m11.411231769s) [upload=14051]: Upload finished. err: <nil> Duration: 5.820934699s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [XXXX........................................................] duration: 6.199842905s FAIL (2025-08-26 08:53:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 08:07:40.970349974 (16h4m8.129115698s) [upload=14051]: Starting upload Aug 26 08:07:40.970414868 (16h4m8.129180602s) [upload=14051]: Done starting upload Aug 26 08:07:40.970581266 (16h4m8.129347001s) [upload=14051]: Uploading segments... Aug 26 08:07:40.970610923 (16h4m8.129376647s) [upload=14051]: Got next segment. Inline: false Aug 26 08:07:40.970630972 (16h4m8.129396706s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 08:07:40.970634669 (16h4m8.129400403s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 08:07:41.325086070 (16h4m8.483851804s) [seg_pos={0 0}, upload=14051]: Begin upload segment... Aug 26 08:07:41.325126537 (16h4m8.483892271s) [seg_pos={0 0}, upload=14051]: Done begin upload segment. Aug 26 08:07:41.325149130 (16h4m8.483914854s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325190900 (16h4m8.483956634s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325532844 (16h4m8.484298578s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325708391 (16h4m8.484474115s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325913813 (16h4m8.484679547s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326099950 (16h4m8.484865684s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326210281 (16h4m8.484976015s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326446914 (16h4m8.485212648s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326495045 (16h4m8.485260779s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326778057 (16h4m8.485543791s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326803586 (16h4m8.48556932s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327065918 (16h4m8.485831652s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327156781 (16h4m8.485922515s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327280669 (16h4m8.486046403s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327439733 (16h4m8.486205467s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327490521 (16h4m8.486256255s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:43.233257592 (16h4m10.392023326s) [upload=14051]: Next returned nil segment Aug 26 08:07:43.233285976 (16h4m10.39205171s) [upload=14051]: Waiting for error group managing segments... Aug 26 08:07:43.691090233 (16h4m10.849855967s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.691200514 (16h4m10.849966248s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.704377618 (16h4m10.863143342s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.743908296 (16h4m10.90267403s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.751936938 (16h4m10.910702673s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.752355369 (16h4m10.911121103s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.791779473 (16h4m10.950545207s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.806507966 (16h4m10.96527369s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.809165941 (16h4m10.967931675s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.842162998 (16h4m11.000928732s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.848984690 (16h4m11.007750424s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.881281698 (16h4m11.040047432s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915580876 (16h4m11.07434661s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915616184 (16h4m11.074381918s) [seg_pos={0 0}, upload=14051]: Segment reached optimal threshold of 13 pieces. Aug 26 08:07:43.915943981 (16h4m11.074709705s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Aug 26 08:07:43.915976523 (16h4m11.074742368s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection Aug 26 08:07:43.916020267 (16h4m11.074786101s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection Aug 26 08:07:43.916083849 (16h4m11.074849583s) [seg_pos={0 0}, upload=14051]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 08:07:43.916183800 (16h4m11.074949544s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 08:07:43.916190372 (16h4m11.074956136s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 08:07:44.252445185 (16h4m11.411210919s) [upload=14051]: Done uploading segments... Aug 26 08:07:44.252466035 (16h4m11.411231769s) [upload=14051]: Upload finished. err: <nil> Duration: 6.199869114s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [XXXXX.......................................................] duration: 5.918676639s FAIL (2025-08-26 08:53:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 08:07:40.970349974 (16h4m8.129115698s) [upload=14051]: Starting upload Aug 26 08:07:40.970414868 (16h4m8.129180602s) [upload=14051]: Done starting upload Aug 26 08:07:40.970581266 (16h4m8.129347001s) [upload=14051]: Uploading segments... Aug 26 08:07:40.970610923 (16h4m8.129376647s) [upload=14051]: Got next segment. Inline: false Aug 26 08:07:40.970630972 (16h4m8.129396706s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 08:07:40.970634669 (16h4m8.129400403s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 08:07:41.325086070 (16h4m8.483851804s) [seg_pos={0 0}, upload=14051]: Begin upload segment... Aug 26 08:07:41.325126537 (16h4m8.483892271s) [seg_pos={0 0}, upload=14051]: Done begin upload segment. Aug 26 08:07:41.325149130 (16h4m8.483914854s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325190900 (16h4m8.483956634s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325532844 (16h4m8.484298578s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325708391 (16h4m8.484474115s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325913813 (16h4m8.484679547s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326099950 (16h4m8.484865684s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326210281 (16h4m8.484976015s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326446914 (16h4m8.485212648s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326495045 (16h4m8.485260779s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326778057 (16h4m8.485543791s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326803586 (16h4m8.48556932s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327065918 (16h4m8.485831652s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327156781 (16h4m8.485922515s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327280669 (16h4m8.486046403s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327439733 (16h4m8.486205467s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327490521 (16h4m8.486256255s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:43.233257592 (16h4m10.392023326s) [upload=14051]: Next returned nil segment Aug 26 08:07:43.233285976 (16h4m10.39205171s) [upload=14051]: Waiting for error group managing segments... Aug 26 08:07:43.691090233 (16h4m10.849855967s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.691200514 (16h4m10.849966248s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.704377618 (16h4m10.863143342s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.743908296 (16h4m10.90267403s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.751936938 (16h4m10.910702673s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.752355369 (16h4m10.911121103s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.791779473 (16h4m10.950545207s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.806507966 (16h4m10.96527369s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.809165941 (16h4m10.967931675s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.842162998 (16h4m11.000928732s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.848984690 (16h4m11.007750424s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.881281698 (16h4m11.040047432s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915580876 (16h4m11.07434661s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915616184 (16h4m11.074381918s) [seg_pos={0 0}, upload=14051]: Segment reached optimal threshold of 13 pieces. Aug 26 08:07:43.915943981 (16h4m11.074709705s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Aug 26 08:07:43.915976523 (16h4m11.074742368s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection Aug 26 08:07:43.916020267 (16h4m11.074786101s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection Aug 26 08:07:43.916083849 (16h4m11.074849583s) [seg_pos={0 0}, upload=14051]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 08:07:43.916183800 (16h4m11.074949544s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 08:07:43.916190372 (16h4m11.074956136s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 08:07:44.252445185 (16h4m11.411210919s) [upload=14051]: Done uploading segments... Aug 26 08:07:44.252466035 (16h4m11.411231769s) [upload=14051]: Upload finished. err: <nil> Duration: 5.91871367s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [XXXXXX......................................................] duration: 5.968371837s FAIL (2025-08-26 08:51:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 08:07:40.970349974 (16h4m8.129115698s) [upload=14051]: Starting upload Aug 26 08:07:40.970414868 (16h4m8.129180602s) [upload=14051]: Done starting upload Aug 26 08:07:40.970581266 (16h4m8.129347001s) [upload=14051]: Uploading segments... Aug 26 08:07:40.970610923 (16h4m8.129376647s) [upload=14051]: Got next segment. Inline: false Aug 26 08:07:40.970630972 (16h4m8.129396706s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 08:07:40.970634669 (16h4m8.129400403s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 08:07:41.325086070 (16h4m8.483851804s) [seg_pos={0 0}, upload=14051]: Begin upload segment... Aug 26 08:07:41.325126537 (16h4m8.483892271s) [seg_pos={0 0}, upload=14051]: Done begin upload segment. Aug 26 08:07:41.325149130 (16h4m8.483914854s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325190900 (16h4m8.483956634s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325532844 (16h4m8.484298578s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325708391 (16h4m8.484474115s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325913813 (16h4m8.484679547s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326099950 (16h4m8.484865684s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326210281 (16h4m8.484976015s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326446914 (16h4m8.485212648s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326495045 (16h4m8.485260779s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326778057 (16h4m8.485543791s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326803586 (16h4m8.48556932s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327065918 (16h4m8.485831652s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327156781 (16h4m8.485922515s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327280669 (16h4m8.486046403s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327439733 (16h4m8.486205467s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327490521 (16h4m8.486256255s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:43.233257592 (16h4m10.392023326s) [upload=14051]: Next returned nil segment Aug 26 08:07:43.233285976 (16h4m10.39205171s) [upload=14051]: Waiting for error group managing segments... Aug 26 08:07:43.691090233 (16h4m10.849855967s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.691200514 (16h4m10.849966248s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.704377618 (16h4m10.863143342s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.743908296 (16h4m10.90267403s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.751936938 (16h4m10.910702673s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.752355369 (16h4m10.911121103s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.791779473 (16h4m10.950545207s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.806507966 (16h4m10.96527369s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.809165941 (16h4m10.967931675s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.842162998 (16h4m11.000928732s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.848984690 (16h4m11.007750424s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.881281698 (16h4m11.040047432s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915580876 (16h4m11.07434661s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915616184 (16h4m11.074381918s) [seg_pos={0 0}, upload=14051]: Segment reached optimal threshold of 13 pieces. Aug 26 08:07:43.915943981 (16h4m11.074709705s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Aug 26 08:07:43.915976523 (16h4m11.074742368s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection Aug 26 08:07:43.916020267 (16h4m11.074786101s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection Aug 26 08:07:43.916083849 (16h4m11.074849583s) [seg_pos={0 0}, upload=14051]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 08:07:43.916183800 (16h4m11.074949544s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 08:07:43.916190372 (16h4m11.074956136s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 08:07:44.252445185 (16h4m11.411210919s) [upload=14051]: Done uploading segments... Aug 26 08:07:44.252466035 (16h4m11.411231769s) [upload=14051]: Upload finished. err: <nil> Duration: 5.96861948s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [XXXXXXX.....................................................] duration: 5.888078313s FAIL (2025-08-26 08:51:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 08:07:40.970349974 (16h4m8.129115698s) [upload=14051]: Starting upload Aug 26 08:07:40.970414868 (16h4m8.129180602s) [upload=14051]: Done starting upload Aug 26 08:07:40.970581266 (16h4m8.129347001s) [upload=14051]: Uploading segments... Aug 26 08:07:40.970610923 (16h4m8.129376647s) [upload=14051]: Got next segment. Inline: false Aug 26 08:07:40.970630972 (16h4m8.129396706s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 08:07:40.970634669 (16h4m8.129400403s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 08:07:41.325086070 (16h4m8.483851804s) [seg_pos={0 0}, upload=14051]: Begin upload segment... Aug 26 08:07:41.325126537 (16h4m8.483892271s) [seg_pos={0 0}, upload=14051]: Done begin upload segment. Aug 26 08:07:41.325149130 (16h4m8.483914854s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325190900 (16h4m8.483956634s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325532844 (16h4m8.484298578s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325708391 (16h4m8.484474115s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325913813 (16h4m8.484679547s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326099950 (16h4m8.484865684s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326210281 (16h4m8.484976015s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326446914 (16h4m8.485212648s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326495045 (16h4m8.485260779s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326778057 (16h4m8.485543791s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326803586 (16h4m8.48556932s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327065918 (16h4m8.485831652s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327156781 (16h4m8.485922515s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327280669 (16h4m8.486046403s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327439733 (16h4m8.486205467s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327490521 (16h4m8.486256255s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:43.233257592 (16h4m10.392023326s) [upload=14051]: Next returned nil segment Aug 26 08:07:43.233285976 (16h4m10.39205171s) [upload=14051]: Waiting for error group managing segments... Aug 26 08:07:43.691090233 (16h4m10.849855967s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.691200514 (16h4m10.849966248s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.704377618 (16h4m10.863143342s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.743908296 (16h4m10.90267403s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.751936938 (16h4m10.910702673s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.752355369 (16h4m10.911121103s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.791779473 (16h4m10.950545207s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.806507966 (16h4m10.96527369s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.809165941 (16h4m10.967931675s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.842162998 (16h4m11.000928732s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.848984690 (16h4m11.007750424s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.881281698 (16h4m11.040047432s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915580876 (16h4m11.07434661s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915616184 (16h4m11.074381918s) [seg_pos={0 0}, upload=14051]: Segment reached optimal threshold of 13 pieces. Aug 26 08:07:43.915943981 (16h4m11.074709705s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Aug 26 08:07:43.915976523 (16h4m11.074742368s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection Aug 26 08:07:43.916020267 (16h4m11.074786101s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection Aug 26 08:07:43.916083849 (16h4m11.074849583s) [seg_pos={0 0}, upload=14051]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 08:07:43.916183800 (16h4m11.074949544s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 08:07:43.916190372 (16h4m11.074956136s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 08:07:44.252445185 (16h4m11.411210919s) [upload=14051]: Done uploading segments... Aug 26 08:07:44.252466035 (16h4m11.411231769s) [upload=14051]: Upload finished. err: <nil> Duration: 5.888103641s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [XXXXXXXX....................................................] duration: 5.899661014s FAIL (2025-08-26 08:50:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 08:07:40.970349974 (16h4m8.129115698s) [upload=14051]: Starting upload Aug 26 08:07:40.970414868 (16h4m8.129180602s) [upload=14051]: Done starting upload Aug 26 08:07:40.970581266 (16h4m8.129347001s) [upload=14051]: Uploading segments... Aug 26 08:07:40.970610923 (16h4m8.129376647s) [upload=14051]: Got next segment. Inline: false Aug 26 08:07:40.970630972 (16h4m8.129396706s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 08:07:40.970634669 (16h4m8.129400403s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 08:07:41.325086070 (16h4m8.483851804s) [seg_pos={0 0}, upload=14051]: Begin upload segment... Aug 26 08:07:41.325126537 (16h4m8.483892271s) [seg_pos={0 0}, upload=14051]: Done begin upload segment. Aug 26 08:07:41.325149130 (16h4m8.483914854s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325190900 (16h4m8.483956634s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325532844 (16h4m8.484298578s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325708391 (16h4m8.484474115s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325913813 (16h4m8.484679547s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326099950 (16h4m8.484865684s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326210281 (16h4m8.484976015s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326446914 (16h4m8.485212648s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326495045 (16h4m8.485260779s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326778057 (16h4m8.485543791s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326803586 (16h4m8.48556932s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327065918 (16h4m8.485831652s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327156781 (16h4m8.485922515s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327280669 (16h4m8.486046403s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327439733 (16h4m8.486205467s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327490521 (16h4m8.486256255s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:43.233257592 (16h4m10.392023326s) [upload=14051]: Next returned nil segment Aug 26 08:07:43.233285976 (16h4m10.39205171s) [upload=14051]: Waiting for error group managing segments... Aug 26 08:07:43.691090233 (16h4m10.849855967s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.691200514 (16h4m10.849966248s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.704377618 (16h4m10.863143342s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.743908296 (16h4m10.90267403s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.751936938 (16h4m10.910702673s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.752355369 (16h4m10.911121103s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.791779473 (16h4m10.950545207s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.806507966 (16h4m10.96527369s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.809165941 (16h4m10.967931675s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.842162998 (16h4m11.000928732s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.848984690 (16h4m11.007750424s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.881281698 (16h4m11.040047432s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915580876 (16h4m11.07434661s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915616184 (16h4m11.074381918s) [seg_pos={0 0}, upload=14051]: Segment reached optimal threshold of 13 pieces. Aug 26 08:07:43.915943981 (16h4m11.074709705s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Aug 26 08:07:43.915976523 (16h4m11.074742368s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection Aug 26 08:07:43.916020267 (16h4m11.074786101s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection Aug 26 08:07:43.916083849 (16h4m11.074849583s) [seg_pos={0 0}, upload=14051]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 08:07:43.916183800 (16h4m11.074949544s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 08:07:43.916190372 (16h4m11.074956136s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 08:07:44.252445185 (16h4m11.411210919s) [upload=14051]: Done uploading segments... Aug 26 08:07:44.252466035 (16h4m11.411231769s) [upload=14051]: Upload finished. err: <nil> Duration: 5.899690079s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [XXXXXXXXX...................................................] duration: 5.861406886s FAIL (2025-08-26 08:49:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 08:07:40.970349974 (16h4m8.129115698s) [upload=14051]: Starting upload Aug 26 08:07:40.970414868 (16h4m8.129180602s) [upload=14051]: Done starting upload Aug 26 08:07:40.970581266 (16h4m8.129347001s) [upload=14051]: Uploading segments... Aug 26 08:07:40.970610923 (16h4m8.129376647s) [upload=14051]: Got next segment. Inline: false Aug 26 08:07:40.970630972 (16h4m8.129396706s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 08:07:40.970634669 (16h4m8.129400403s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 08:07:41.325086070 (16h4m8.483851804s) [seg_pos={0 0}, upload=14051]: Begin upload segment... Aug 26 08:07:41.325126537 (16h4m8.483892271s) [seg_pos={0 0}, upload=14051]: Done begin upload segment. Aug 26 08:07:41.325149130 (16h4m8.483914854s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325190900 (16h4m8.483956634s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325532844 (16h4m8.484298578s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325708391 (16h4m8.484474115s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325913813 (16h4m8.484679547s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326099950 (16h4m8.484865684s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326210281 (16h4m8.484976015s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326446914 (16h4m8.485212648s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326495045 (16h4m8.485260779s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326778057 (16h4m8.485543791s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326803586 (16h4m8.48556932s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327065918 (16h4m8.485831652s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327156781 (16h4m8.485922515s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327280669 (16h4m8.486046403s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327439733 (16h4m8.486205467s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327490521 (16h4m8.486256255s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:43.233257592 (16h4m10.392023326s) [upload=14051]: Next returned nil segment Aug 26 08:07:43.233285976 (16h4m10.39205171s) [upload=14051]: Waiting for error group managing segments... Aug 26 08:07:43.691090233 (16h4m10.849855967s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.691200514 (16h4m10.849966248s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.704377618 (16h4m10.863143342s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.743908296 (16h4m10.90267403s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.751936938 (16h4m10.910702673s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.752355369 (16h4m10.911121103s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.791779473 (16h4m10.950545207s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.806507966 (16h4m10.96527369s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.809165941 (16h4m10.967931675s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.842162998 (16h4m11.000928732s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.848984690 (16h4m11.007750424s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.881281698 (16h4m11.040047432s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915580876 (16h4m11.07434661s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915616184 (16h4m11.074381918s) [seg_pos={0 0}, upload=14051]: Segment reached optimal threshold of 13 pieces. Aug 26 08:07:43.915943981 (16h4m11.074709705s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Aug 26 08:07:43.915976523 (16h4m11.074742368s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection Aug 26 08:07:43.916020267 (16h4m11.074786101s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection Aug 26 08:07:43.916083849 (16h4m11.074849583s) [seg_pos={0 0}, upload=14051]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 08:07:43.916183800 (16h4m11.074949544s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 08:07:43.916190372 (16h4m11.074956136s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 08:07:44.252445185 (16h4m11.411210919s) [upload=14051]: Done uploading segments... Aug 26 08:07:44.252466035 (16h4m11.411231769s) [upload=14051]: Upload finished. err: <nil> Duration: 5.861508079s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [XXXXXXXXXX..................................................] duration: 5.910564397s FAIL (2025-08-26 08:49:16 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 08:07:40.970349974 (16h4m8.129115698s) [upload=14051]: Starting upload Aug 26 08:07:40.970414868 (16h4m8.129180602s) [upload=14051]: Done starting upload Aug 26 08:07:40.970581266 (16h4m8.129347001s) [upload=14051]: Uploading segments... Aug 26 08:07:40.970610923 (16h4m8.129376647s) [upload=14051]: Got next segment. Inline: false Aug 26 08:07:40.970630972 (16h4m8.129396706s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 08:07:40.970634669 (16h4m8.129400403s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 08:07:41.325086070 (16h4m8.483851804s) [seg_pos={0 0}, upload=14051]: Begin upload segment... Aug 26 08:07:41.325126537 (16h4m8.483892271s) [seg_pos={0 0}, upload=14051]: Done begin upload segment. Aug 26 08:07:41.325149130 (16h4m8.483914854s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325190900 (16h4m8.483956634s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325532844 (16h4m8.484298578s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325708391 (16h4m8.484474115s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325913813 (16h4m8.484679547s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326099950 (16h4m8.484865684s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326210281 (16h4m8.484976015s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326446914 (16h4m8.485212648s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326495045 (16h4m8.485260779s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326778057 (16h4m8.485543791s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326803586 (16h4m8.48556932s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327065918 (16h4m8.485831652s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327156781 (16h4m8.485922515s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327280669 (16h4m8.486046403s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327439733 (16h4m8.486205467s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327490521 (16h4m8.486256255s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:43.233257592 (16h4m10.392023326s) [upload=14051]: Next returned nil segment Aug 26 08:07:43.233285976 (16h4m10.39205171s) [upload=14051]: Waiting for error group managing segments... Aug 26 08:07:43.691090233 (16h4m10.849855967s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.691200514 (16h4m10.849966248s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.704377618 (16h4m10.863143342s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.743908296 (16h4m10.90267403s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.751936938 (16h4m10.910702673s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.752355369 (16h4m10.911121103s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.791779473 (16h4m10.950545207s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.806507966 (16h4m10.96527369s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.809165941 (16h4m10.967931675s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.842162998 (16h4m11.000928732s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.848984690 (16h4m11.007750424s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.881281698 (16h4m11.040047432s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915580876 (16h4m11.07434661s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915616184 (16h4m11.074381918s) [seg_pos={0 0}, upload=14051]: Segment reached optimal threshold of 13 pieces. Aug 26 08:07:43.915943981 (16h4m11.074709705s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Aug 26 08:07:43.915976523 (16h4m11.074742368s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection Aug 26 08:07:43.916020267 (16h4m11.074786101s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection Aug 26 08:07:43.916083849 (16h4m11.074849583s) [seg_pos={0 0}, upload=14051]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 08:07:43.916183800 (16h4m11.074949544s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 08:07:43.916190372 (16h4m11.074956136s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 08:07:44.252445185 (16h4m11.411210919s) [upload=14051]: Done uploading segments... Aug 26 08:07:44.252466035 (16h4m11.411231769s) [upload=14051]: Upload finished. err: <nil> Duration: 5.910591759s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [XXXXXXXXXXX.................................................] duration: 5.861084331s FAIL (2025-08-26 08:48:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 08:07:40.970349974 (16h4m8.129115698s) [upload=14051]: Starting upload Aug 26 08:07:40.970414868 (16h4m8.129180602s) [upload=14051]: Done starting upload Aug 26 08:07:40.970581266 (16h4m8.129347001s) [upload=14051]: Uploading segments... Aug 26 08:07:40.970610923 (16h4m8.129376647s) [upload=14051]: Got next segment. Inline: false Aug 26 08:07:40.970630972 (16h4m8.129396706s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 08:07:40.970634669 (16h4m8.129400403s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 08:07:41.325086070 (16h4m8.483851804s) [seg_pos={0 0}, upload=14051]: Begin upload segment... Aug 26 08:07:41.325126537 (16h4m8.483892271s) [seg_pos={0 0}, upload=14051]: Done begin upload segment. Aug 26 08:07:41.325149130 (16h4m8.483914854s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325190900 (16h4m8.483956634s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325532844 (16h4m8.484298578s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325708391 (16h4m8.484474115s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325913813 (16h4m8.484679547s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326099950 (16h4m8.484865684s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326210281 (16h4m8.484976015s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326446914 (16h4m8.485212648s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326495045 (16h4m8.485260779s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326778057 (16h4m8.485543791s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326803586 (16h4m8.48556932s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327065918 (16h4m8.485831652s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327156781 (16h4m8.485922515s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327280669 (16h4m8.486046403s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327439733 (16h4m8.486205467s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327490521 (16h4m8.486256255s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:43.233257592 (16h4m10.392023326s) [upload=14051]: Next returned nil segment Aug 26 08:07:43.233285976 (16h4m10.39205171s) [upload=14051]: Waiting for error group managing segments... Aug 26 08:07:43.691090233 (16h4m10.849855967s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.691200514 (16h4m10.849966248s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.704377618 (16h4m10.863143342s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.743908296 (16h4m10.90267403s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.751936938 (16h4m10.910702673s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.752355369 (16h4m10.911121103s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.791779473 (16h4m10.950545207s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.806507966 (16h4m10.96527369s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.809165941 (16h4m10.967931675s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.842162998 (16h4m11.000928732s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.848984690 (16h4m11.007750424s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.881281698 (16h4m11.040047432s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915580876 (16h4m11.07434661s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915616184 (16h4m11.074381918s) [seg_pos={0 0}, upload=14051]: Segment reached optimal threshold of 13 pieces. Aug 26 08:07:43.915943981 (16h4m11.074709705s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Aug 26 08:07:43.915976523 (16h4m11.074742368s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection Aug 26 08:07:43.916020267 (16h4m11.074786101s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection Aug 26 08:07:43.916083849 (16h4m11.074849583s) [seg_pos={0 0}, upload=14051]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 08:07:43.916183800 (16h4m11.074949544s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 08:07:43.916190372 (16h4m11.074956136s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 08:07:44.252445185 (16h4m11.411210919s) [upload=14051]: Done uploading segments... Aug 26 08:07:44.252466035 (16h4m11.411231769s) [upload=14051]: Upload finished. err: <nil> Duration: 5.861099441s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [XXXXXXXXXXXX................................................] duration: 5.909466505s FAIL (2025-08-26 08:47:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 08:07:40.970349974 (16h4m8.129115698s) [upload=14051]: Starting upload Aug 26 08:07:40.970414868 (16h4m8.129180602s) [upload=14051]: Done starting upload Aug 26 08:07:40.970581266 (16h4m8.129347001s) [upload=14051]: Uploading segments... Aug 26 08:07:40.970610923 (16h4m8.129376647s) [upload=14051]: Got next segment. Inline: false Aug 26 08:07:40.970630972 (16h4m8.129396706s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 08:07:40.970634669 (16h4m8.129400403s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 08:07:41.325086070 (16h4m8.483851804s) [seg_pos={0 0}, upload=14051]: Begin upload segment... Aug 26 08:07:41.325126537 (16h4m8.483892271s) [seg_pos={0 0}, upload=14051]: Done begin upload segment. Aug 26 08:07:41.325149130 (16h4m8.483914854s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325190900 (16h4m8.483956634s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325532844 (16h4m8.484298578s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325708391 (16h4m8.484474115s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325913813 (16h4m8.484679547s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326099950 (16h4m8.484865684s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326210281 (16h4m8.484976015s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326446914 (16h4m8.485212648s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326495045 (16h4m8.485260779s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326778057 (16h4m8.485543791s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326803586 (16h4m8.48556932s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327065918 (16h4m8.485831652s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327156781 (16h4m8.485922515s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327280669 (16h4m8.486046403s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327439733 (16h4m8.486205467s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327490521 (16h4m8.486256255s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:43.233257592 (16h4m10.392023326s) [upload=14051]: Next returned nil segment Aug 26 08:07:43.233285976 (16h4m10.39205171s) [upload=14051]: Waiting for error group managing segments... Aug 26 08:07:43.691090233 (16h4m10.849855967s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.691200514 (16h4m10.849966248s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.704377618 (16h4m10.863143342s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.743908296 (16h4m10.90267403s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.751936938 (16h4m10.910702673s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.752355369 (16h4m10.911121103s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.791779473 (16h4m10.950545207s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.806507966 (16h4m10.96527369s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.809165941 (16h4m10.967931675s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.842162998 (16h4m11.000928732s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.848984690 (16h4m11.007750424s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.881281698 (16h4m11.040047432s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915580876 (16h4m11.07434661s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915616184 (16h4m11.074381918s) [seg_pos={0 0}, upload=14051]: Segment reached optimal threshold of 13 pieces. Aug 26 08:07:43.915943981 (16h4m11.074709705s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Aug 26 08:07:43.915976523 (16h4m11.074742368s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection Aug 26 08:07:43.916020267 (16h4m11.074786101s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection Aug 26 08:07:43.916083849 (16h4m11.074849583s) [seg_pos={0 0}, upload=14051]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 08:07:43.916183800 (16h4m11.074949544s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 08:07:43.916190372 (16h4m11.074956136s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 08:07:44.252445185 (16h4m11.411210919s) [upload=14051]: Done uploading segments... Aug 26 08:07:44.252466035 (16h4m11.411231769s) [upload=14051]: Upload finished. err: <nil> Duration: 5.909492245s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [XXXXXXXXXXXXX...............................................] duration: 6.136213935s FAIL (2025-08-26 08:47:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 08:07:40.970349974 (16h4m8.129115698s) [upload=14051]: Starting upload Aug 26 08:07:40.970414868 (16h4m8.129180602s) [upload=14051]: Done starting upload Aug 26 08:07:40.970581266 (16h4m8.129347001s) [upload=14051]: Uploading segments... Aug 26 08:07:40.970610923 (16h4m8.129376647s) [upload=14051]: Got next segment. Inline: false Aug 26 08:07:40.970630972 (16h4m8.129396706s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 08:07:40.970634669 (16h4m8.129400403s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 08:07:41.325086070 (16h4m8.483851804s) [seg_pos={0 0}, upload=14051]: Begin upload segment... Aug 26 08:07:41.325126537 (16h4m8.483892271s) [seg_pos={0 0}, upload=14051]: Done begin upload segment. Aug 26 08:07:41.325149130 (16h4m8.483914854s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325190900 (16h4m8.483956634s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325532844 (16h4m8.484298578s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325708391 (16h4m8.484474115s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325913813 (16h4m8.484679547s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326099950 (16h4m8.484865684s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326210281 (16h4m8.484976015s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326446914 (16h4m8.485212648s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326495045 (16h4m8.485260779s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326778057 (16h4m8.485543791s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326803586 (16h4m8.48556932s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327065918 (16h4m8.485831652s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327156781 (16h4m8.485922515s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327280669 (16h4m8.486046403s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327439733 (16h4m8.486205467s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327490521 (16h4m8.486256255s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:43.233257592 (16h4m10.392023326s) [upload=14051]: Next returned nil segment Aug 26 08:07:43.233285976 (16h4m10.39205171s) [upload=14051]: Waiting for error group managing segments... Aug 26 08:07:43.691090233 (16h4m10.849855967s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.691200514 (16h4m10.849966248s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.704377618 (16h4m10.863143342s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.743908296 (16h4m10.90267403s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.751936938 (16h4m10.910702673s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.752355369 (16h4m10.911121103s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.791779473 (16h4m10.950545207s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.806507966 (16h4m10.96527369s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.809165941 (16h4m10.967931675s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.842162998 (16h4m11.000928732s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.848984690 (16h4m11.007750424s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.881281698 (16h4m11.040047432s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915580876 (16h4m11.07434661s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915616184 (16h4m11.074381918s) [seg_pos={0 0}, upload=14051]: Segment reached optimal threshold of 13 pieces. Aug 26 08:07:43.915943981 (16h4m11.074709705s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Aug 26 08:07:43.915976523 (16h4m11.074742368s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection Aug 26 08:07:43.916020267 (16h4m11.074786101s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection Aug 26 08:07:43.916083849 (16h4m11.074849583s) [seg_pos={0 0}, upload=14051]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 08:07:43.916183800 (16h4m11.074949544s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 08:07:43.916190372 (16h4m11.074956136s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 08:07:44.252445185 (16h4m11.411210919s) [upload=14051]: Done uploading segments... Aug 26 08:07:44.252466035 (16h4m11.411231769s) [upload=14051]: Upload finished. err: <nil> Duration: 6.13623821s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXX..............................................] duration: 6.128626819s FAIL (2025-08-26 08:46:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 08:07:40.970349974 (16h4m8.129115698s) [upload=14051]: Starting upload Aug 26 08:07:40.970414868 (16h4m8.129180602s) [upload=14051]: Done starting upload Aug 26 08:07:40.970581266 (16h4m8.129347001s) [upload=14051]: Uploading segments... Aug 26 08:07:40.970610923 (16h4m8.129376647s) [upload=14051]: Got next segment. Inline: false Aug 26 08:07:40.970630972 (16h4m8.129396706s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 08:07:40.970634669 (16h4m8.129400403s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 08:07:41.325086070 (16h4m8.483851804s) [seg_pos={0 0}, upload=14051]: Begin upload segment... Aug 26 08:07:41.325126537 (16h4m8.483892271s) [seg_pos={0 0}, upload=14051]: Done begin upload segment. Aug 26 08:07:41.325149130 (16h4m8.483914854s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325190900 (16h4m8.483956634s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325532844 (16h4m8.484298578s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325708391 (16h4m8.484474115s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325913813 (16h4m8.484679547s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326099950 (16h4m8.484865684s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326210281 (16h4m8.484976015s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326446914 (16h4m8.485212648s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326495045 (16h4m8.485260779s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326778057 (16h4m8.485543791s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326803586 (16h4m8.48556932s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327065918 (16h4m8.485831652s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327156781 (16h4m8.485922515s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327280669 (16h4m8.486046403s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327439733 (16h4m8.486205467s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327490521 (16h4m8.486256255s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:43.233257592 (16h4m10.392023326s) [upload=14051]: Next returned nil segment Aug 26 08:07:43.233285976 (16h4m10.39205171s) [upload=14051]: Waiting for error group managing segments... Aug 26 08:07:43.691090233 (16h4m10.849855967s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.691200514 (16h4m10.849966248s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.704377618 (16h4m10.863143342s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.743908296 (16h4m10.90267403s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.751936938 (16h4m10.910702673s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.752355369 (16h4m10.911121103s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.791779473 (16h4m10.950545207s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.806507966 (16h4m10.96527369s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.809165941 (16h4m10.967931675s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.842162998 (16h4m11.000928732s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.848984690 (16h4m11.007750424s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.881281698 (16h4m11.040047432s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915580876 (16h4m11.07434661s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915616184 (16h4m11.074381918s) [seg_pos={0 0}, upload=14051]: Segment reached optimal threshold of 13 pieces. Aug 26 08:07:43.915943981 (16h4m11.074709705s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Aug 26 08:07:43.915976523 (16h4m11.074742368s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection Aug 26 08:07:43.916020267 (16h4m11.074786101s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection Aug 26 08:07:43.916083849 (16h4m11.074849583s) [seg_pos={0 0}, upload=14051]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 08:07:43.916183800 (16h4m11.074949544s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 08:07:43.916190372 (16h4m11.074956136s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 08:07:44.252445185 (16h4m11.411210919s) [upload=14051]: Done uploading segments... Aug 26 08:07:44.252466035 (16h4m11.411231769s) [upload=14051]: Upload finished. err: <nil> Duration: 6.128662707s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [X..............] duration: 383.416302ms target: https://www.storj.io/ FAIL (2025-08-26 08:45:55 UTC) orbiter warning: Get "https://www.storj.io/": dial tcp 13.233.175.166:443: i/o timeout Duration: 383.445297ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [.X.............] duration: 380.52613ms target: https://www.storj.io/ FAIL (2025-08-26 08:45:29 UTC) orbiter warning: Get "https://www.storj.io/": dial tcp 13.233.175.166:443: i/o timeout Duration: 380.550598ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXX.............................................] duration: 5.993352956s FAIL (2025-08-26 08:44:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 08:07:40.970349974 (16h4m8.129115698s) [upload=14051]: Starting upload Aug 26 08:07:40.970414868 (16h4m8.129180602s) [upload=14051]: Done starting upload Aug 26 08:07:40.970581266 (16h4m8.129347001s) [upload=14051]: Uploading segments... Aug 26 08:07:40.970610923 (16h4m8.129376647s) [upload=14051]: Got next segment. Inline: false Aug 26 08:07:40.970630972 (16h4m8.129396706s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 08:07:40.970634669 (16h4m8.129400403s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 08:07:41.325086070 (16h4m8.483851804s) [seg_pos={0 0}, upload=14051]: Begin upload segment... Aug 26 08:07:41.325126537 (16h4m8.483892271s) [seg_pos={0 0}, upload=14051]: Done begin upload segment. Aug 26 08:07:41.325149130 (16h4m8.483914854s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325190900 (16h4m8.483956634s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325532844 (16h4m8.484298578s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325708391 (16h4m8.484474115s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325913813 (16h4m8.484679547s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326099950 (16h4m8.484865684s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326210281 (16h4m8.484976015s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326446914 (16h4m8.485212648s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326495045 (16h4m8.485260779s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326778057 (16h4m8.485543791s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326803586 (16h4m8.48556932s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327065918 (16h4m8.485831652s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327156781 (16h4m8.485922515s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327280669 (16h4m8.486046403s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327439733 (16h4m8.486205467s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327490521 (16h4m8.486256255s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:43.233257592 (16h4m10.392023326s) [upload=14051]: Next returned nil segment Aug 26 08:07:43.233285976 (16h4m10.39205171s) [upload=14051]: Waiting for error group managing segments... Aug 26 08:07:43.691090233 (16h4m10.849855967s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.691200514 (16h4m10.849966248s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.704377618 (16h4m10.863143342s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.743908296 (16h4m10.90267403s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.751936938 (16h4m10.910702673s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.752355369 (16h4m10.911121103s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.791779473 (16h4m10.950545207s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.806507966 (16h4m10.96527369s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.809165941 (16h4m10.967931675s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.842162998 (16h4m11.000928732s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.848984690 (16h4m11.007750424s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.881281698 (16h4m11.040047432s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915580876 (16h4m11.07434661s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915616184 (16h4m11.074381918s) [seg_pos={0 0}, upload=14051]: Segment reached optimal threshold of 13 pieces. Aug 26 08:07:43.915943981 (16h4m11.074709705s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Aug 26 08:07:43.915976523 (16h4m11.074742368s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection Aug 26 08:07:43.916020267 (16h4m11.074786101s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection Aug 26 08:07:43.916083849 (16h4m11.074849583s) [seg_pos={0 0}, upload=14051]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 08:07:43.916183800 (16h4m11.074949544s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 08:07:43.916190372 (16h4m11.074956136s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 08:07:44.252445185 (16h4m11.411210919s) [upload=14051]: Done uploading segments... Aug 26 08:07:44.252466035 (16h4m11.411231769s) [upload=14051]: Upload finished. err: <nil> Duration: 5.993378184s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [..X............] duration: 433.123231ms target: https://www.storj.io/ FAIL (2025-08-26 08:44:50 UTC) orbiter warning: Get "https://www.storj.io/": dial tcp 13.233.175.166:443: i/o timeout Duration: 433.147427ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXX............................................] duration: 5.897724098s FAIL (2025-08-26 08:43:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 08:07:40.970349974 (16h4m8.129115698s) [upload=14051]: Starting upload Aug 26 08:07:40.970414868 (16h4m8.129180602s) [upload=14051]: Done starting upload Aug 26 08:07:40.970581266 (16h4m8.129347001s) [upload=14051]: Uploading segments... Aug 26 08:07:40.970610923 (16h4m8.129376647s) [upload=14051]: Got next segment. Inline: false Aug 26 08:07:40.970630972 (16h4m8.129396706s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 08:07:40.970634669 (16h4m8.129400403s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 08:07:41.325086070 (16h4m8.483851804s) [seg_pos={0 0}, upload=14051]: Begin upload segment... Aug 26 08:07:41.325126537 (16h4m8.483892271s) [seg_pos={0 0}, upload=14051]: Done begin upload segment. Aug 26 08:07:41.325149130 (16h4m8.483914854s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325190900 (16h4m8.483956634s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325532844 (16h4m8.484298578s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325708391 (16h4m8.484474115s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325913813 (16h4m8.484679547s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326099950 (16h4m8.484865684s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326210281 (16h4m8.484976015s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326446914 (16h4m8.485212648s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326495045 (16h4m8.485260779s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326778057 (16h4m8.485543791s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326803586 (16h4m8.48556932s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327065918 (16h4m8.485831652s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327156781 (16h4m8.485922515s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327280669 (16h4m8.486046403s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327439733 (16h4m8.486205467s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327490521 (16h4m8.486256255s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:43.233257592 (16h4m10.392023326s) [upload=14051]: Next returned nil segment Aug 26 08:07:43.233285976 (16h4m10.39205171s) [upload=14051]: Waiting for error group managing segments... Aug 26 08:07:43.691090233 (16h4m10.849855967s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.691200514 (16h4m10.849966248s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.704377618 (16h4m10.863143342s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.743908296 (16h4m10.90267403s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.751936938 (16h4m10.910702673s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.752355369 (16h4m10.911121103s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.791779473 (16h4m10.950545207s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.806507966 (16h4m10.96527369s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.809165941 (16h4m10.967931675s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.842162998 (16h4m11.000928732s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.848984690 (16h4m11.007750424s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.881281698 (16h4m11.040047432s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915580876 (16h4m11.07434661s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915616184 (16h4m11.074381918s) [seg_pos={0 0}, upload=14051]: Segment reached optimal threshold of 13 pieces. Aug 26 08:07:43.915943981 (16h4m11.074709705s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Aug 26 08:07:43.915976523 (16h4m11.074742368s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection Aug 26 08:07:43.916020267 (16h4m11.074786101s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection Aug 26 08:07:43.916083849 (16h4m11.074849583s) [seg_pos={0 0}, upload=14051]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 08:07:43.916183800 (16h4m11.074949544s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 08:07:43.916190372 (16h4m11.074956136s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 08:07:44.252445185 (16h4m11.411210919s) [upload=14051]: Done uploading segments... Aug 26 08:07:44.252466035 (16h4m11.411231769s) [upload=14051]: Upload finished. err: <nil> Duration: 5.897758243s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [...X...........] duration: 476.421127ms target: https://www.storj.io/ FAIL (2025-08-26 08:43:37 UTC) orbiter warning: Get "https://www.storj.io/": dial tcp 13.233.175.166:443: i/o timeout Duration: 476.4492ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXX...........................................] duration: 6.275577937s FAIL (2025-08-26 08:43:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 08:07:40.970349974 (16h4m8.129115698s) [upload=14051]: Starting upload Aug 26 08:07:40.970414868 (16h4m8.129180602s) [upload=14051]: Done starting upload Aug 26 08:07:40.970581266 (16h4m8.129347001s) [upload=14051]: Uploading segments... Aug 26 08:07:40.970610923 (16h4m8.129376647s) [upload=14051]: Got next segment. Inline: false Aug 26 08:07:40.970630972 (16h4m8.129396706s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 08:07:40.970634669 (16h4m8.129400403s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 08:07:41.325086070 (16h4m8.483851804s) [seg_pos={0 0}, upload=14051]: Begin upload segment... Aug 26 08:07:41.325126537 (16h4m8.483892271s) [seg_pos={0 0}, upload=14051]: Done begin upload segment. Aug 26 08:07:41.325149130 (16h4m8.483914854s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325190900 (16h4m8.483956634s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325532844 (16h4m8.484298578s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325708391 (16h4m8.484474115s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325913813 (16h4m8.484679547s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326099950 (16h4m8.484865684s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326210281 (16h4m8.484976015s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326446914 (16h4m8.485212648s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326495045 (16h4m8.485260779s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326778057 (16h4m8.485543791s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326803586 (16h4m8.48556932s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327065918 (16h4m8.485831652s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327156781 (16h4m8.485922515s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327280669 (16h4m8.486046403s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327439733 (16h4m8.486205467s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327490521 (16h4m8.486256255s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:43.233257592 (16h4m10.392023326s) [upload=14051]: Next returned nil segment Aug 26 08:07:43.233285976 (16h4m10.39205171s) [upload=14051]: Waiting for error group managing segments... Aug 26 08:07:43.691090233 (16h4m10.849855967s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.691200514 (16h4m10.849966248s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.704377618 (16h4m10.863143342s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.743908296 (16h4m10.90267403s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.751936938 (16h4m10.910702673s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.752355369 (16h4m10.911121103s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.791779473 (16h4m10.950545207s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.806507966 (16h4m10.96527369s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.809165941 (16h4m10.967931675s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.842162998 (16h4m11.000928732s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.848984690 (16h4m11.007750424s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.881281698 (16h4m11.040047432s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915580876 (16h4m11.07434661s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915616184 (16h4m11.074381918s) [seg_pos={0 0}, upload=14051]: Segment reached optimal threshold of 13 pieces. Aug 26 08:07:43.915943981 (16h4m11.074709705s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Aug 26 08:07:43.915976523 (16h4m11.074742368s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection Aug 26 08:07:43.916020267 (16h4m11.074786101s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection Aug 26 08:07:43.916083849 (16h4m11.074849583s) [seg_pos={0 0}, upload=14051]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 08:07:43.916183800 (16h4m11.074949544s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 08:07:43.916190372 (16h4m11.074956136s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 08:07:44.252445185 (16h4m11.411210919s) [upload=14051]: Done uploading segments... Aug 26 08:07:44.252466035 (16h4m11.411231769s) [upload=14051]: Upload finished. err: <nil> Duration: 6.275622462s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [....X..........] duration: 736.927061ms target: https://www.storj.io/ FAIL (2025-08-26 08:43:14 UTC) orbiter warning: Get "https://www.storj.io/": dial tcp 13.233.175.166:443: i/o timeout Duration: 736.955856ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXX..........................................] duration: 5.932663852s FAIL (2025-08-26 08:42:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 08:07:40.970349974 (16h4m8.129115698s) [upload=14051]: Starting upload Aug 26 08:07:40.970414868 (16h4m8.129180602s) [upload=14051]: Done starting upload Aug 26 08:07:40.970581266 (16h4m8.129347001s) [upload=14051]: Uploading segments... Aug 26 08:07:40.970610923 (16h4m8.129376647s) [upload=14051]: Got next segment. Inline: false Aug 26 08:07:40.970630972 (16h4m8.129396706s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 08:07:40.970634669 (16h4m8.129400403s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 08:07:41.325086070 (16h4m8.483851804s) [seg_pos={0 0}, upload=14051]: Begin upload segment... Aug 26 08:07:41.325126537 (16h4m8.483892271s) [seg_pos={0 0}, upload=14051]: Done begin upload segment. Aug 26 08:07:41.325149130 (16h4m8.483914854s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325190900 (16h4m8.483956634s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325532844 (16h4m8.484298578s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325708391 (16h4m8.484474115s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325913813 (16h4m8.484679547s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326099950 (16h4m8.484865684s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326210281 (16h4m8.484976015s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326446914 (16h4m8.485212648s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326495045 (16h4m8.485260779s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326778057 (16h4m8.485543791s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326803586 (16h4m8.48556932s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327065918 (16h4m8.485831652s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327156781 (16h4m8.485922515s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327280669 (16h4m8.486046403s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327439733 (16h4m8.486205467s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327490521 (16h4m8.486256255s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:43.233257592 (16h4m10.392023326s) [upload=14051]: Next returned nil segment Aug 26 08:07:43.233285976 (16h4m10.39205171s) [upload=14051]: Waiting for error group managing segments... Aug 26 08:07:43.691090233 (16h4m10.849855967s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.691200514 (16h4m10.849966248s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.704377618 (16h4m10.863143342s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.743908296 (16h4m10.90267403s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.751936938 (16h4m10.910702673s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.752355369 (16h4m10.911121103s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.791779473 (16h4m10.950545207s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.806507966 (16h4m10.96527369s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.809165941 (16h4m10.967931675s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.842162998 (16h4m11.000928732s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.848984690 (16h4m11.007750424s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.881281698 (16h4m11.040047432s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915580876 (16h4m11.07434661s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915616184 (16h4m11.074381918s) [seg_pos={0 0}, upload=14051]: Segment reached optimal threshold of 13 pieces. Aug 26 08:07:43.915943981 (16h4m11.074709705s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Aug 26 08:07:43.915976523 (16h4m11.074742368s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection Aug 26 08:07:43.916020267 (16h4m11.074786101s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection Aug 26 08:07:43.916083849 (16h4m11.074849583s) [seg_pos={0 0}, upload=14051]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 08:07:43.916183800 (16h4m11.074949544s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 08:07:43.916190372 (16h4m11.074956136s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 08:07:44.252445185 (16h4m11.411210919s) [upload=14051]: Done uploading segments... Aug 26 08:07:44.252466035 (16h4m11.411231769s) [upload=14051]: Upload finished. err: <nil> Duration: 5.93269372s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [.....X.........] duration: 430.456409ms target: https://www.storj.io/ FAIL (2025-08-26 08:42:14 UTC) orbiter warning: Get "https://www.storj.io/": dial tcp 13.233.175.166:443: i/o timeout Duration: 430.486988ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [......X........] duration: 386.949462ms target: https://www.storj.io/ FAIL (2025-08-26 08:41:50 UTC) orbiter warning: Get "https://www.storj.io/": dial tcp 13.233.175.166:443: i/o timeout Duration: 386.975222ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 19 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXX.........................................] duration: 5.946330833s FAIL (2025-08-26 08:41:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 08:07:40.970349974 (16h4m8.129115698s) [upload=14051]: Starting upload Aug 26 08:07:40.970414868 (16h4m8.129180602s) [upload=14051]: Done starting upload Aug 26 08:07:40.970581266 (16h4m8.129347001s) [upload=14051]: Uploading segments... Aug 26 08:07:40.970610923 (16h4m8.129376647s) [upload=14051]: Got next segment. Inline: false Aug 26 08:07:40.970630972 (16h4m8.129396706s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 08:07:40.970634669 (16h4m8.129400403s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 08:07:41.325086070 (16h4m8.483851804s) [seg_pos={0 0}, upload=14051]: Begin upload segment... Aug 26 08:07:41.325126537 (16h4m8.483892271s) [seg_pos={0 0}, upload=14051]: Done begin upload segment. Aug 26 08:07:41.325149130 (16h4m8.483914854s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325190900 (16h4m8.483956634s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325532844 (16h4m8.484298578s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325708391 (16h4m8.484474115s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325913813 (16h4m8.484679547s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326099950 (16h4m8.484865684s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326210281 (16h4m8.484976015s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326446914 (16h4m8.485212648s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326495045 (16h4m8.485260779s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326778057 (16h4m8.485543791s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326803586 (16h4m8.48556932s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327065918 (16h4m8.485831652s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327156781 (16h4m8.485922515s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327280669 (16h4m8.486046403s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327439733 (16h4m8.486205467s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327490521 (16h4m8.486256255s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:43.233257592 (16h4m10.392023326s) [upload=14051]: Next returned nil segment Aug 26 08:07:43.233285976 (16h4m10.39205171s) [upload=14051]: Waiting for error group managing segments... Aug 26 08:07:43.691090233 (16h4m10.849855967s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.691200514 (16h4m10.849966248s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.704377618 (16h4m10.863143342s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.743908296 (16h4m10.90267403s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.751936938 (16h4m10.910702673s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.752355369 (16h4m10.911121103s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.791779473 (16h4m10.950545207s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.806507966 (16h4m10.96527369s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.809165941 (16h4m10.967931675s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.842162998 (16h4m11.000928732s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.848984690 (16h4m11.007750424s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.881281698 (16h4m11.040047432s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915580876 (16h4m11.07434661s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915616184 (16h4m11.074381918s) [seg_pos={0 0}, upload=14051]: Segment reached optimal threshold of 13 pieces. Aug 26 08:07:43.915943981 (16h4m11.074709705s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Aug 26 08:07:43.915976523 (16h4m11.074742368s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection Aug 26 08:07:43.916020267 (16h4m11.074786101s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection Aug 26 08:07:43.916083849 (16h4m11.074849583s) [seg_pos={0 0}, upload=14051]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 08:07:43.916183800 (16h4m11.074949544s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 08:07:43.916190372 (16h4m11.074956136s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 08:07:44.252445185 (16h4m11.411210919s) [upload=14051]: Done uploading segments... Aug 26 08:07:44.252466035 (16h4m11.411231769s) [upload=14051]: Upload finished. err: <nil> Duration: 5.946359879s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [.......X.......] duration: 398.717919ms target: https://www.storj.io/ FAIL (2025-08-26 08:41:06 UTC) orbiter warning: Get "https://www.storj.io/": dial tcp 13.233.175.166:443: i/o timeout Duration: 398.746043ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 20 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXX........................................] duration: 5.900355869s FAIL (2025-08-26 08:40:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 08:07:40.970349974 (16h4m8.129115698s) [upload=14051]: Starting upload Aug 26 08:07:40.970414868 (16h4m8.129180602s) [upload=14051]: Done starting upload Aug 26 08:07:40.970581266 (16h4m8.129347001s) [upload=14051]: Uploading segments... Aug 26 08:07:40.970610923 (16h4m8.129376647s) [upload=14051]: Got next segment. Inline: false Aug 26 08:07:40.970630972 (16h4m8.129396706s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 08:07:40.970634669 (16h4m8.129400403s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 08:07:41.325086070 (16h4m8.483851804s) [seg_pos={0 0}, upload=14051]: Begin upload segment... Aug 26 08:07:41.325126537 (16h4m8.483892271s) [seg_pos={0 0}, upload=14051]: Done begin upload segment. Aug 26 08:07:41.325149130 (16h4m8.483914854s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325190900 (16h4m8.483956634s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325532844 (16h4m8.484298578s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325708391 (16h4m8.484474115s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325913813 (16h4m8.484679547s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326099950 (16h4m8.484865684s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326210281 (16h4m8.484976015s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326446914 (16h4m8.485212648s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326495045 (16h4m8.485260779s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326778057 (16h4m8.485543791s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326803586 (16h4m8.48556932s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327065918 (16h4m8.485831652s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327156781 (16h4m8.485922515s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327280669 (16h4m8.486046403s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327439733 (16h4m8.486205467s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327490521 (16h4m8.486256255s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:43.233257592 (16h4m10.392023326s) [upload=14051]: Next returned nil segment Aug 26 08:07:43.233285976 (16h4m10.39205171s) [upload=14051]: Waiting for error group managing segments... Aug 26 08:07:43.691090233 (16h4m10.849855967s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.691200514 (16h4m10.849966248s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.704377618 (16h4m10.863143342s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.743908296 (16h4m10.90267403s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.751936938 (16h4m10.910702673s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.752355369 (16h4m10.911121103s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.791779473 (16h4m10.950545207s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.806507966 (16h4m10.96527369s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.809165941 (16h4m10.967931675s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.842162998 (16h4m11.000928732s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.848984690 (16h4m11.007750424s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.881281698 (16h4m11.040047432s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915580876 (16h4m11.07434661s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915616184 (16h4m11.074381918s) [seg_pos={0 0}, upload=14051]: Segment reached optimal threshold of 13 pieces. Aug 26 08:07:43.915943981 (16h4m11.074709705s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Aug 26 08:07:43.915976523 (16h4m11.074742368s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection Aug 26 08:07:43.916020267 (16h4m11.074786101s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection Aug 26 08:07:43.916083849 (16h4m11.074849583s) [seg_pos={0 0}, upload=14051]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 08:07:43.916183800 (16h4m11.074949544s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 08:07:43.916190372 (16h4m11.074956136s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 08:07:44.252445185 (16h4m11.411210919s) [upload=14051]: Done uploading segments... Aug 26 08:07:44.252466035 (16h4m11.411231769s) [upload=14051]: Upload finished. err: <nil> Duration: 5.900383001s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [........X......] duration: 710.088126ms target: https://www.storj.io/ FAIL (2025-08-26 08:39:55 UTC) orbiter warning: Get "https://www.storj.io/": dial tcp 13.233.175.166:443: i/o timeout Duration: 710.116712ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXX.......................................] duration: 5.975389798s FAIL (2025-08-26 08:39:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 08:07:40.970349974 (16h4m8.129115698s) [upload=14051]: Starting upload Aug 26 08:07:40.970414868 (16h4m8.129180602s) [upload=14051]: Done starting upload Aug 26 08:07:40.970581266 (16h4m8.129347001s) [upload=14051]: Uploading segments... Aug 26 08:07:40.970610923 (16h4m8.129376647s) [upload=14051]: Got next segment. Inline: false Aug 26 08:07:40.970630972 (16h4m8.129396706s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 08:07:40.970634669 (16h4m8.129400403s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 08:07:41.325086070 (16h4m8.483851804s) [seg_pos={0 0}, upload=14051]: Begin upload segment... Aug 26 08:07:41.325126537 (16h4m8.483892271s) [seg_pos={0 0}, upload=14051]: Done begin upload segment. Aug 26 08:07:41.325149130 (16h4m8.483914854s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325190900 (16h4m8.483956634s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325532844 (16h4m8.484298578s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325708391 (16h4m8.484474115s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325913813 (16h4m8.484679547s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326099950 (16h4m8.484865684s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326210281 (16h4m8.484976015s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326446914 (16h4m8.485212648s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326495045 (16h4m8.485260779s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326778057 (16h4m8.485543791s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326803586 (16h4m8.48556932s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327065918 (16h4m8.485831652s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327156781 (16h4m8.485922515s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327280669 (16h4m8.486046403s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327439733 (16h4m8.486205467s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327490521 (16h4m8.486256255s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:43.233257592 (16h4m10.392023326s) [upload=14051]: Next returned nil segment Aug 26 08:07:43.233285976 (16h4m10.39205171s) [upload=14051]: Waiting for error group managing segments... Aug 26 08:07:43.691090233 (16h4m10.849855967s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.691200514 (16h4m10.849966248s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.704377618 (16h4m10.863143342s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.743908296 (16h4m10.90267403s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.751936938 (16h4m10.910702673s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.752355369 (16h4m10.911121103s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.791779473 (16h4m10.950545207s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.806507966 (16h4m10.96527369s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.809165941 (16h4m10.967931675s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.842162998 (16h4m11.000928732s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.848984690 (16h4m11.007750424s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.881281698 (16h4m11.040047432s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915580876 (16h4m11.07434661s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915616184 (16h4m11.074381918s) [seg_pos={0 0}, upload=14051]: Segment reached optimal threshold of 13 pieces. Aug 26 08:07:43.915943981 (16h4m11.074709705s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Aug 26 08:07:43.915976523 (16h4m11.074742368s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection Aug 26 08:07:43.916020267 (16h4m11.074786101s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection Aug 26 08:07:43.916083849 (16h4m11.074849583s) [seg_pos={0 0}, upload=14051]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 08:07:43.916183800 (16h4m11.074949544s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 08:07:43.916190372 (16h4m11.074956136s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 08:07:44.252445185 (16h4m11.411210919s) [upload=14051]: Done uploading segments... Aug 26 08:07:44.252466035 (16h4m11.411231769s) [upload=14051]: Upload finished. err: <nil> Duration: 5.975416539s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [.........X.....] duration: 378.514401ms target: https://www.storj.io/ FAIL (2025-08-26 08:38:52 UTC) orbiter warning: Get "https://www.storj.io/": dial tcp 13.233.175.166:443: i/o timeout Duration: 378.543577ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [..........X....] duration: 618.885379ms target: https://www.storj.io/ FAIL (2025-08-26 08:38:36 UTC) orbiter warning: Get "https://www.storj.io/": dial tcp 13.233.175.166:443: i/o timeout Duration: 618.903644ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXX......................................] duration: 5.890018418s FAIL (2025-08-26 08:38:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 26 08:07:40.970349974 (16h4m8.129115698s) [upload=14051]: Starting upload Aug 26 08:07:40.970414868 (16h4m8.129180602s) [upload=14051]: Done starting upload Aug 26 08:07:40.970581266 (16h4m8.129347001s) [upload=14051]: Uploading segments... Aug 26 08:07:40.970610923 (16h4m8.129376647s) [upload=14051]: Got next segment. Inline: false Aug 26 08:07:40.970630972 (16h4m8.129396706s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 08:07:40.970634669 (16h4m8.129400403s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 08:07:41.325086070 (16h4m8.483851804s) [seg_pos={0 0}, upload=14051]: Begin upload segment... Aug 26 08:07:41.325126537 (16h4m8.483892271s) [seg_pos={0 0}, upload=14051]: Done begin upload segment. Aug 26 08:07:41.325149130 (16h4m8.483914854s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325190900 (16h4m8.483956634s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325532844 (16h4m8.484298578s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325708391 (16h4m8.484474115s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.325913813 (16h4m8.484679547s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326099950 (16h4m8.484865684s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326210281 (16h4m8.484976015s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326446914 (16h4m8.485212648s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326495045 (16h4m8.485260779s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326778057 (16h4m8.485543791s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.326803586 (16h4m8.48556932s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327065918 (16h4m8.485831652s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327156781 (16h4m8.485922515s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327280669 (16h4m8.486046403s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327439733 (16h4m8.486205467s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:41.327490521 (16h4m8.486256255s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Uploading piece... Aug 26 08:07:43.233257592 (16h4m10.392023326s) [upload=14051]: Next returned nil segment Aug 26 08:07:43.233285976 (16h4m10.39205171s) [upload=14051]: Waiting for error group managing segments... Aug 26 08:07:43.691090233 (16h4m10.849855967s) [noise=true , address=45.91.116.14:4190 , piece_id=ZPYUA7MHDGQDU4L7IXIKCM4S4YDBZ27MOPIIJ4PDGBHPYQKYF7TQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.691200514 (16h4m10.849966248s) [noise=true , address=45.91.116.8:4045 , piece_id=EOX7VPKOPPUQIMS745XAKNWI47PKYI2LDM72AMJ6EO23IU4SJBFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.704377618 (16h4m10.863143342s) [noise=true , address=77.90.31.14:4662 , piece_id=OZRN4OZQ5MPWB3LPA7UI7LQRN3UGB3747TPHT5I7FMM3ZL2VWKQQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.743908296 (16h4m10.90267403s) [noise=true , address=77.90.31.12:4592 , piece_id=GSSIZCD4KBZGKDKBTKPIF2VWDGG2YK5YW632XQ4MRK4GDNWM2CSA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.751936938 (16h4m10.910702673s) [noise=true , address=77.90.31.10:4518 , piece_id=GBHI6Y5LLHVBO45H7UG6AGWXE2ZJYHSOZVNHUGFPKH7OC62E37BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.752355369 (16h4m10.911121103s) [noise=true , address=45.91.116.6:4975 , piece_id=IQTYDP33XF44WKUTAXPYF6ITZWVNWZAP2SFC6DLUMXSG6RWHQBCA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.791779473 (16h4m10.950545207s) [noise=true , address=77.90.31.15:4697 , piece_id=WTEOB5FICT5KGWEEIX6GPJ2YF6GBCDGP7ZRQSCK4PBVFFXPHJ25Q, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.806507966 (16h4m10.96527369s) [noise=true , address=45.91.116.4:4939 , piece_id=65VPFXCXYOKE5AVAOPHFWDTVLJLCKM5QFB3PUCZ4SXGX7VPYXY4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.809165941 (16h4m10.967931675s) [noise=true , address=45.91.116.11:4118 , piece_id=FVRAX4BDQBF4CSQPOLRWNS5VS7RV6HEK53T2FOP7NGIJ5ZZGMSBA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.842162998 (16h4m11.000928732s) [noise=true , address=104.36.82.6:4875 , piece_id=FAAWT4IDQYHJJC5SI5FYY6RSR72TPBKVUK3IKHA5T3EF5WAMO4GA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.848984690 (16h4m11.007750424s) [noise=true , address=104.36.82.16:4294 , piece_id=BQVWI3MHA33WCPSXOJ3N62ADAONFG6QDF7EN5AU3JLNWSPEWV7BQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.881281698 (16h4m11.040047432s) [noise=true , address=104.36.82.13:4186 , piece_id=4KIJQ6YN7W5Q7KNABIDA3OYWNRLKAXMR7Z5GIHADZ47IE2ZY7TFQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915580876 (16h4m11.07434661s) [noise=true , address=104.36.82.2:4731 , piece_id=QXAKDGLOAML7D2CYCGXMZKENOPFRK2YYWINO3X2PTCT3G7KKZT4A, seg_pos={0 0}, upload=14051]: Done uploading piece. err: <nil> Aug 26 08:07:43.915616184 (16h4m11.074381918s) [seg_pos={0 0}, upload=14051]: Segment reached optimal threshold of 13 pieces. Aug 26 08:07:43.915943981 (16h4m11.074709705s) [noise=true , address=104.36.82.4:4801 , piece_id=RPEK25OA5Z4TGAXNHU2IDAXKKWF6ZHNKC7H42CGRGZMZPEJVCFXQ, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Aug 26 08:07:43.915976523 (16h4m11.074742368s) [noise=true , address=77.90.31.19:4841 , piece_id=MGKZKSHZTCFG6LVS7D7KOLQFIC3ECLQ3D3PBASOTI2VM3MCVKNRA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32800->77.90.31.19:4841: use of closed network connection Aug 26 08:07:43.916020267 (16h4m11.074786101s) [noise=true , address=77.90.31.16:4733 , piece_id=EPIROV766BXBSQJLULNZSJCZDD64BUKBE3PPENVNRFCDQKSXV6CA, seg_pos={0 0}, upload=14051]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60266->77.90.31.16:4733: use of closed network connection Aug 26 08:07:43.916083849 (16h4m11.074849583s) [seg_pos={0 0}, upload=14051]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 08:07:43.916183800 (16h4m11.074949544s) [upload=14051]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 08:07:43.916190372 (16h4m11.074956136s) [upload=14051]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 08:07:44.252445185 (16h4m11.411210919s) [upload=14051]: Done uploading segments... Aug 26 08:07:44.252466035 (16h4m11.411231769s) [upload=14051]: Upload finished. err: <nil> Duration: 5.890046142s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [...........X...] duration: 585.507243ms target: https://www.storj.io/ FAIL (2025-08-26 08:37:40 UTC) orbiter warning: Get "https://www.storj.io/": dial tcp 13.233.175.166:443: i/o timeout Duration: 585.533803ms core website

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)