Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.917393344s trace: https://jaeger.dev.storj.io/trace/574375048ac9530f FAIL (2025-12-07 22:49:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.91742717s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................................................X..] duration: 3.474580478s trace: https://jaeger.dev.storj.io/trace/4cd87cb5652f9c06 FAIL (2025-12-07 22:49:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.474615163s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 4.925076299s trace: https://jaeger.dev.storj.io/trace/696a333689b50b84 FAIL (2025-12-07 22:48:57 UTC) orbiter warning: operation: sync2.Copy uplink: piecestore: (Node ID: 1ByMj85eQ9yd81bpEjSKrqFTEnLsuV132WoAMBxHrd12jgqddx, Piece ID: PK5SS7JPDNWVGZ4EJ43YWL4MNKGSCS7564EM62UJPBRHXKB4KBHA) context deadline exceeded; piecestore: (Node ID: 1LmXgEBf4pEfYehAES7kR3sVLRt7QxS2pvttiNtqaDnYp7PDzy, Piece ID: Z7PBN4D4PZIWZZZDO7A666MCKEUIZ7ZG3O64BVY3YITRZ7SJISEQ) context deadline exceeded; piecestore: (Node ID: 12Hq4E5wWAPzrxZacP6gbFpKiBvbRXnPdSR4z9jRU443yCFqS2r, Piece ID: CAAZO3SF5Y2YQL2VXBEONDARGL3AXYRZWJSFH3CMANA4J5GOHD7Q) context deadline exceeded; piecestore: (Node ID: 12SQdxBgr7pu6Kz3M16tJjezo5bMiG1gKB57kS3PWN3ibsFVEJ8, Piece ID: 5RB4OO2XHF7WVT7HVTEQL4P4HAJYP3WRHDQVFWERDRBNKUVXYKSA) context deadline exceeded; piecestore: (Node ID: 1hz2v7h419qCvgzkvckp6PYDeQRKRGyHsLtyMGeG1VMa58A8EF, Piece ID: 2YOEIDEDQHBSYGLLEYYHV3BL63VCF2X55JPY5TKSBJSPOKUFEHHQ) context deadline exceeded; piecestore: (Node ID: 1uT7ntK6KAVDSErVdc37UcSd7xzCd3Dont23WitCYrA6KN4h4r, Piece ID: IGM33ZIHTUUKUXREFCY22HT7GRRK6RFODU7SU2N5ZXHWVIRO57KA) context deadline exceeded; piecestore: (Node ID: 1Ea8H6e3FPUzq42DSetCZ9e9ZZWtWpqyDy3fdYBucQDcCci8NN, Piece ID: WAZBXZ323RJVVK3IOKEHLLQJFA5LTNPQV72IP3GS3337GH4VGDAQ) context deadline exceeded; piecestore: (Node ID: 12ifo86KtZXZxEbUAxnvmX5tAg1C27mYub4VUTc9CkWTmpzPP3J, Piece ID: E4XT2PLZFP2HNL62PO5PMMXGQLMBDWV6FP63YSKKOV6LILK4EZLQ) context deadline exceeded; piecestore: (Node ID: 1APgLgqBaSndUajVXyuLWvkM2jbxwuYDkJhxVhweWNDjuju5A3, Piece ID: CDKE76XLSEISNM37HMKFG7STBORFP6NN47F326YW7B6HBEBGSZ4A) context deadline exceeded; piecestore: (Node ID: 12k5wzLZYUYDuQcbMv8NiXV2XBuZvbXbio3GErJYkzNEezwMwLG, Piece ID: TY4UXQXVTIAU2EWYBAFFPNM5OAQHJ5BJL24MGT7V5MSCGV4IDQ2Q) context deadline exceeded; piecestore: (Node ID: 12Ae31qNHzgX2cfrD8EVZqvJ9J95hijL4TEYtuGQPSpWNABG4kw, Piece ID: BIQOY7CGH7SDRD3YZHEBFNBIXUTAUXJMKHUOPTZIFSFXQCIVKSRA) context deadline exceeded; piecestore: (Node ID: 12EaBCW6FX8WY7HGhJJvDZrUy4QuBn8mH7xLz3v4BQA4rdKfKEg, Piece ID: ZJKUDMMW5TXZXRM63BHJ2IAOTQVRXQRMCLQFE6NYRQENBCI7VWIA) context deadline exceeded; piecestore: (Node ID: 12QzLS4YZYRfg6Aw82iFPkAioAuMn22Sf2EkwJFiZZsBDdvv1vS, Piece ID: HG7A6G3H3WOHBUO2QXHZZSYCYRPGZ3NW5MI7VE7F64GJHOHUQ34Q) context deadline exceeded; piecestore: (Node ID: 1vSFRya2YG5Agcz3txzeFfEvtymD4qGEBqrZjFYwEBR3L6tSdU, Piece ID: 5CBLLQNEOPVUGWZ3GYIXS65SDMJESBXYPPUUXH55HVRRKDGDYCPA) context deadline exceeded; piecestore: (Node ID: 12Tw2fxHJaQWpMnX3zZRtmp4Rpy9DEGx6vdZdo6x4eC6nvNgj7K, Piece ID: ULAMA6H4CRAUSN6KYQ2XXVJ4BMCJWB7WWTKBPTSQ7HF2IAZOHWNQ) context deadline exceeded; piecestore: (Node ID: 12aNknfsj5LsTiS8qb42nUSX1dYMhKbwesR6sZJiZtRJWzvfyqk, Piece ID: CKQCDBMNHUROXS34YWSXMJI4Z2A7DQHGKFTJBSGZ463YNGO62ZPA) context deadline exceeded; piecestore: (Node ID: 12UyANaqcy2NBjkmrA1buuHKpJcCxVz8vMpBzqujgmTQB78DY4u, Piece ID: POQJTXSQ45VBTFONHUXVLWMIY62MSQL6VZM6OOAHO42YIIZ6JJCA) context deadline exceeded; piecestore: (Node ID: 12YiXJaHTWUBq5fbKoFHHzeVxdSqpzeMb838cZgynEur4xHAEP, Piece ID: 2NRFXZUZJMBOCUXG6ZF2NRA4ILQ34UUPIPXHZE3QXRMAIWPUQTQA) context deadline exceeded; piecestore: (Node ID: 12uLGjR4eCbBC8GY5vPc4UJmj4AkzGxQd7WQeSo5aw4AxbpaF6J, Piece ID: XGCOLCZIHEWZAJHYAX45DH5LISJCNAWTPHRNM6Q2IZ3UC4TRDDZA) context deadline exceeded; piecestore: (Node ID: 12ntDfA4LBBPkXUioZRWwQhMMA7kBN5rpMd91BgMi7tw2uuRi7n, Piece ID: 4D7CGCOOSMD6WHPV6PRLKDAME6A6SKX3NT32XS2KLYZKO2CJTURA) context deadline exceeded; piecestore: (Node ID: 12QjxocGH8nPtfBmZqTVmV3sssQs5nR8QsWaVga3BdKU97XgPBz, Piece ID: O75ISG3WWM46CC4ZALCC56AAVRXMRPYUMUOMWZVIEWNTLQ5B6UVQ) context deadline exceeded; piecestore: (Node ID: 1qEoNfbFCs5TobVBDG2aV1qSNU6At8ZUxSeGGaYFytgCcArXg7, Piece ID: VNDX43FYBGVJMPQ4773RC5OSYOPTJTYD26RI37ULIRNZKKLATSZA) context deadline exceeded; piecestore: (Node ID: 12hRf4FXCVVxzPjwRGscsH6Ei7Cgo3EQciSDH99PCNonSP9vTVE, Piece ID: YG22O5IGTGUBMAAODU3T7EBKRLRATKLYBNVSRAZ7APWNIAL2NXKA) context deadline exceeded; piecestore: (Node ID: 15KQ2SjAnXeXBccMHNKZff2PLjeLd2a1WNFtk1ebx1GBdhd3kw, Piece ID: VZWPUBPHQRLPE67H7PPWBO4LGQ7ANOZAFJY3P6VIGQEYJZI5DYYA) context deadline exceeded; piecestore: (Node ID: 12eEQzjo5uQZGL1JR2AyinqCXxderwuqTKvTmoFZ3uNDqwE6WAo, Piece ID: E2SFJCYFLH6UQZ5BZN2YOB3NKEMLRGAJB2GPIXIUBBK53P7VNDOQ) context deadline exceeded; piecestore: (Node ID: 12vQvbGahDxH7tm6NWygYvfuCerQaLqA3BQNEG7bSDKgEKRbUaJ, Piece ID: YTRN7GBUWG5QQIYBU4RB7VK7LS2PTRXTV44BN7P5GCO2VF5F5D2A) context deadline exceeded; piecestore: (Node ID: 12XCy3WvXLNmgSisHzPVy8MFWS8jiA5gnTNTiux7s4FaLnaJWAU, Piece ID: ZD4QR2TJPVJLE545HQGYDR5NDZPEWUMQISU3A2PNTVBE7WH6AU6A) context deadline exceeded Duration: 4.925097371s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................................................X.] duration: 3.503610777s trace: https://jaeger.dev.storj.io/trace/4cd87cb5652f9c06 FAIL (2025-12-07 22:48:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.503637769s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.734060537s trace: https://jaeger.dev.storj.io/trace/574375048ac9530f FAIL (2025-12-07 22:48:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.73412567s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................................................X] duration: 5.001707769s trace: https://jaeger.dev.storj.io/trace/4cd87cb5652f9c06 FAIL (2025-12-07 22:48:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001732496s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 4.955619969s trace: https://jaeger.dev.storj.io/trace/696a333689b50b84 FAIL (2025-12-07 22:48:01 UTC) orbiter warning: operation: sync2.Copy uplink: piecestore: (Node ID: 1ByMj85eQ9yd81bpEjSKrqFTEnLsuV132WoAMBxHrd12jgqddx, Piece ID: PK5SS7JPDNWVGZ4EJ43YWL4MNKGSCS7564EM62UJPBRHXKB4KBHA) context deadline exceeded; piecestore: (Node ID: 1LmXgEBf4pEfYehAES7kR3sVLRt7QxS2pvttiNtqaDnYp7PDzy, Piece ID: Z7PBN4D4PZIWZZZDO7A666MCKEUIZ7ZG3O64BVY3YITRZ7SJISEQ) context deadline exceeded; piecestore: (Node ID: 12Hq4E5wWAPzrxZacP6gbFpKiBvbRXnPdSR4z9jRU443yCFqS2r, Piece ID: CAAZO3SF5Y2YQL2VXBEONDARGL3AXYRZWJSFH3CMANA4J5GOHD7Q) context deadline exceeded; piecestore: (Node ID: 12SQdxBgr7pu6Kz3M16tJjezo5bMiG1gKB57kS3PWN3ibsFVEJ8, Piece ID: 5RB4OO2XHF7WVT7HVTEQL4P4HAJYP3WRHDQVFWERDRBNKUVXYKSA) context deadline exceeded; piecestore: (Node ID: 1hz2v7h419qCvgzkvckp6PYDeQRKRGyHsLtyMGeG1VMa58A8EF, Piece ID: 2YOEIDEDQHBSYGLLEYYHV3BL63VCF2X55JPY5TKSBJSPOKUFEHHQ) context deadline exceeded; piecestore: (Node ID: 1uT7ntK6KAVDSErVdc37UcSd7xzCd3Dont23WitCYrA6KN4h4r, Piece ID: IGM33ZIHTUUKUXREFCY22HT7GRRK6RFODU7SU2N5ZXHWVIRO57KA) context deadline exceeded; piecestore: (Node ID: 1Ea8H6e3FPUzq42DSetCZ9e9ZZWtWpqyDy3fdYBucQDcCci8NN, Piece ID: WAZBXZ323RJVVK3IOKEHLLQJFA5LTNPQV72IP3GS3337GH4VGDAQ) context deadline exceeded; piecestore: (Node ID: 12ifo86KtZXZxEbUAxnvmX5tAg1C27mYub4VUTc9CkWTmpzPP3J, Piece ID: E4XT2PLZFP2HNL62PO5PMMXGQLMBDWV6FP63YSKKOV6LILK4EZLQ) context deadline exceeded; piecestore: (Node ID: 1APgLgqBaSndUajVXyuLWvkM2jbxwuYDkJhxVhweWNDjuju5A3, Piece ID: CDKE76XLSEISNM37HMKFG7STBORFP6NN47F326YW7B6HBEBGSZ4A) context deadline exceeded; piecestore: (Node ID: 12k5wzLZYUYDuQcbMv8NiXV2XBuZvbXbio3GErJYkzNEezwMwLG, Piece ID: TY4UXQXVTIAU2EWYBAFFPNM5OAQHJ5BJL24MGT7V5MSCGV4IDQ2Q) context deadline exceeded; piecestore: (Node ID: 12Ae31qNHzgX2cfrD8EVZqvJ9J95hijL4TEYtuGQPSpWNABG4kw, Piece ID: BIQOY7CGH7SDRD3YZHEBFNBIXUTAUXJMKHUOPTZIFSFXQCIVKSRA) context deadline exceeded; piecestore: (Node ID: 12EaBCW6FX8WY7HGhJJvDZrUy4QuBn8mH7xLz3v4BQA4rdKfKEg, Piece ID: ZJKUDMMW5TXZXRM63BHJ2IAOTQVRXQRMCLQFE6NYRQENBCI7VWIA) context deadline exceeded; piecestore: (Node ID: 12QzLS4YZYRfg6Aw82iFPkAioAuMn22Sf2EkwJFiZZsBDdvv1vS, Piece ID: HG7A6G3H3WOHBUO2QXHZZSYCYRPGZ3NW5MI7VE7F64GJHOHUQ34Q) context deadline exceeded; piecestore: (Node ID: 1vSFRya2YG5Agcz3txzeFfEvtymD4qGEBqrZjFYwEBR3L6tSdU, Piece ID: 5CBLLQNEOPVUGWZ3GYIXS65SDMJESBXYPPUUXH55HVRRKDGDYCPA) context deadline exceeded; piecestore: (Node ID: 12Tw2fxHJaQWpMnX3zZRtmp4Rpy9DEGx6vdZdo6x4eC6nvNgj7K, Piece ID: ULAMA6H4CRAUSN6KYQ2XXVJ4BMCJWB7WWTKBPTSQ7HF2IAZOHWNQ) context deadline exceeded; piecestore: (Node ID: 12aNknfsj5LsTiS8qb42nUSX1dYMhKbwesR6sZJiZtRJWzvfyqk, Piece ID: CKQCDBMNHUROXS34YWSXMJI4Z2A7DQHGKFTJBSGZ463YNGO62ZPA) context deadline exceeded; piecestore: (Node ID: 12UyANaqcy2NBjkmrA1buuHKpJcCxVz8vMpBzqujgmTQB78DY4u, Piece ID: POQJTXSQ45VBTFONHUXVLWMIY62MSQL6VZM6OOAHO42YIIZ6JJCA) context deadline exceeded; piecestore: (Node ID: 12YiXJaHTWUBq5fbKoFHHzeVxdSqpzeMb838cZgynEur4xHAEP, Piece ID: 2NRFXZUZJMBOCUXG6ZF2NRA4ILQ34UUPIPXHZE3QXRMAIWPUQTQA) context deadline exceeded; piecestore: (Node ID: 12uLGjR4eCbBC8GY5vPc4UJmj4AkzGxQd7WQeSo5aw4AxbpaF6J, Piece ID: XGCOLCZIHEWZAJHYAX45DH5LISJCNAWTPHRNM6Q2IZ3UC4TRDDZA) context deadline exceeded; piecestore: (Node ID: 12ntDfA4LBBPkXUioZRWwQhMMA7kBN5rpMd91BgMi7tw2uuRi7n, Piece ID: 4D7CGCOOSMD6WHPV6PRLKDAME6A6SKX3NT32XS2KLYZKO2CJTURA) context deadline exceeded; piecestore: (Node ID: 12QjxocGH8nPtfBmZqTVmV3sssQs5nR8QsWaVga3BdKU97XgPBz, Piece ID: O75ISG3WWM46CC4ZALCC56AAVRXMRPYUMUOMWZVIEWNTLQ5B6UVQ) context deadline exceeded; piecestore: (Node ID: 1qEoNfbFCs5TobVBDG2aV1qSNU6At8ZUxSeGGaYFytgCcArXg7, Piece ID: VNDX43FYBGVJMPQ4773RC5OSYOPTJTYD26RI37ULIRNZKKLATSZA) context deadline exceeded; piecestore: (Node ID: 12hRf4FXCVVxzPjwRGscsH6Ei7Cgo3EQciSDH99PCNonSP9vTVE, Piece ID: YG22O5IGTGUBMAAODU3T7EBKRLRATKLYBNVSRAZ7APWNIAL2NXKA) context deadline exceeded; piecestore: (Node ID: 15KQ2SjAnXeXBccMHNKZff2PLjeLd2a1WNFtk1ebx1GBdhd3kw, Piece ID: VZWPUBPHQRLPE67H7PPWBO4LGQ7ANOZAFJY3P6VIGQEYJZI5DYYA) context deadline exceeded; piecestore: (Node ID: 12eEQzjo5uQZGL1JR2AyinqCXxderwuqTKvTmoFZ3uNDqwE6WAo, Piece ID: E2SFJCYFLH6UQZ5BZN2YOB3NKEMLRGAJB2GPIXIUBBK53P7VNDOQ) context deadline exceeded; piecestore: (Node ID: 12vQvbGahDxH7tm6NWygYvfuCerQaLqA3BQNEG7bSDKgEKRbUaJ, Piece ID: YTRN7GBUWG5QQIYBU4RB7VK7LS2PTRXTV44BN7P5GCO2VF5F5D2A) context deadline exceeded; piecestore: (Node ID: 12XCy3WvXLNmgSisHzPVy8MFWS8jiA5gnTNTiux7s4FaLnaJWAU, Piece ID: ZD4QR2TJPVJLE545HQGYDR5NDZPEWUMQISU3A2PNTVBE7WH6AU6A) context deadline exceeded Duration: 4.955645837s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 5.052219219s trace: https://jaeger.dev.storj.io/trace/696a333689b50b84 FAIL (2025-12-07 22:47:21 UTC) orbiter warning: operation: sync2.Copy uplink: piecestore: (Node ID: 1ByMj85eQ9yd81bpEjSKrqFTEnLsuV132WoAMBxHrd12jgqddx, Piece ID: PK5SS7JPDNWVGZ4EJ43YWL4MNKGSCS7564EM62UJPBRHXKB4KBHA) context deadline exceeded; piecestore: (Node ID: 1LmXgEBf4pEfYehAES7kR3sVLRt7QxS2pvttiNtqaDnYp7PDzy, Piece ID: Z7PBN4D4PZIWZZZDO7A666MCKEUIZ7ZG3O64BVY3YITRZ7SJISEQ) context deadline exceeded; piecestore: (Node ID: 12Hq4E5wWAPzrxZacP6gbFpKiBvbRXnPdSR4z9jRU443yCFqS2r, Piece ID: CAAZO3SF5Y2YQL2VXBEONDARGL3AXYRZWJSFH3CMANA4J5GOHD7Q) context deadline exceeded; piecestore: (Node ID: 12SQdxBgr7pu6Kz3M16tJjezo5bMiG1gKB57kS3PWN3ibsFVEJ8, Piece ID: 5RB4OO2XHF7WVT7HVTEQL4P4HAJYP3WRHDQVFWERDRBNKUVXYKSA) context deadline exceeded; piecestore: (Node ID: 1hz2v7h419qCvgzkvckp6PYDeQRKRGyHsLtyMGeG1VMa58A8EF, Piece ID: 2YOEIDEDQHBSYGLLEYYHV3BL63VCF2X55JPY5TKSBJSPOKUFEHHQ) context deadline exceeded; piecestore: (Node ID: 1uT7ntK6KAVDSErVdc37UcSd7xzCd3Dont23WitCYrA6KN4h4r, Piece ID: IGM33ZIHTUUKUXREFCY22HT7GRRK6RFODU7SU2N5ZXHWVIRO57KA) context deadline exceeded; piecestore: (Node ID: 1Ea8H6e3FPUzq42DSetCZ9e9ZZWtWpqyDy3fdYBucQDcCci8NN, Piece ID: WAZBXZ323RJVVK3IOKEHLLQJFA5LTNPQV72IP3GS3337GH4VGDAQ) context deadline exceeded; piecestore: (Node ID: 12ifo86KtZXZxEbUAxnvmX5tAg1C27mYub4VUTc9CkWTmpzPP3J, Piece ID: E4XT2PLZFP2HNL62PO5PMMXGQLMBDWV6FP63YSKKOV6LILK4EZLQ) context deadline exceeded; piecestore: (Node ID: 1APgLgqBaSndUajVXyuLWvkM2jbxwuYDkJhxVhweWNDjuju5A3, Piece ID: CDKE76XLSEISNM37HMKFG7STBORFP6NN47F326YW7B6HBEBGSZ4A) context deadline exceeded; piecestore: (Node ID: 12k5wzLZYUYDuQcbMv8NiXV2XBuZvbXbio3GErJYkzNEezwMwLG, Piece ID: TY4UXQXVTIAU2EWYBAFFPNM5OAQHJ5BJL24MGT7V5MSCGV4IDQ2Q) context deadline exceeded; piecestore: (Node ID: 12Ae31qNHzgX2cfrD8EVZqvJ9J95hijL4TEYtuGQPSpWNABG4kw, Piece ID: BIQOY7CGH7SDRD3YZHEBFNBIXUTAUXJMKHUOPTZIFSFXQCIVKSRA) context deadline exceeded; piecestore: (Node ID: 12EaBCW6FX8WY7HGhJJvDZrUy4QuBn8mH7xLz3v4BQA4rdKfKEg, Piece ID: ZJKUDMMW5TXZXRM63BHJ2IAOTQVRXQRMCLQFE6NYRQENBCI7VWIA) context deadline exceeded; piecestore: (Node ID: 12QzLS4YZYRfg6Aw82iFPkAioAuMn22Sf2EkwJFiZZsBDdvv1vS, Piece ID: HG7A6G3H3WOHBUO2QXHZZSYCYRPGZ3NW5MI7VE7F64GJHOHUQ34Q) context deadline exceeded; piecestore: (Node ID: 1vSFRya2YG5Agcz3txzeFfEvtymD4qGEBqrZjFYwEBR3L6tSdU, Piece ID: 5CBLLQNEOPVUGWZ3GYIXS65SDMJESBXYPPUUXH55HVRRKDGDYCPA) context deadline exceeded; piecestore: (Node ID: 12Tw2fxHJaQWpMnX3zZRtmp4Rpy9DEGx6vdZdo6x4eC6nvNgj7K, Piece ID: ULAMA6H4CRAUSN6KYQ2XXVJ4BMCJWB7WWTKBPTSQ7HF2IAZOHWNQ) context deadline exceeded; piecestore: (Node ID: 12aNknfsj5LsTiS8qb42nUSX1dYMhKbwesR6sZJiZtRJWzvfyqk, Piece ID: CKQCDBMNHUROXS34YWSXMJI4Z2A7DQHGKFTJBSGZ463YNGO62ZPA) context deadline exceeded; piecestore: (Node ID: 12UyANaqcy2NBjkmrA1buuHKpJcCxVz8vMpBzqujgmTQB78DY4u, Piece ID: POQJTXSQ45VBTFONHUXVLWMIY62MSQL6VZM6OOAHO42YIIZ6JJCA) context deadline exceeded; piecestore: (Node ID: 12YiXJaHTWUBq5fbKoFHHzeVxdSqpzeMb838cZgynEur4xHAEP, Piece ID: 2NRFXZUZJMBOCUXG6ZF2NRA4ILQ34UUPIPXHZE3QXRMAIWPUQTQA) context deadline exceeded; piecestore: (Node ID: 12uLGjR4eCbBC8GY5vPc4UJmj4AkzGxQd7WQeSo5aw4AxbpaF6J, Piece ID: XGCOLCZIHEWZAJHYAX45DH5LISJCNAWTPHRNM6Q2IZ3UC4TRDDZA) context deadline exceeded; piecestore: (Node ID: 12ntDfA4LBBPkXUioZRWwQhMMA7kBN5rpMd91BgMi7tw2uuRi7n, Piece ID: 4D7CGCOOSMD6WHPV6PRLKDAME6A6SKX3NT32XS2KLYZKO2CJTURA) context deadline exceeded; piecestore: (Node ID: 12QjxocGH8nPtfBmZqTVmV3sssQs5nR8QsWaVga3BdKU97XgPBz, Piece ID: O75ISG3WWM46CC4ZALCC56AAVRXMRPYUMUOMWZVIEWNTLQ5B6UVQ) context deadline exceeded; piecestore: (Node ID: 1qEoNfbFCs5TobVBDG2aV1qSNU6At8ZUxSeGGaYFytgCcArXg7, Piece ID: VNDX43FYBGVJMPQ4773RC5OSYOPTJTYD26RI37ULIRNZKKLATSZA) context deadline exceeded; piecestore: (Node ID: 12hRf4FXCVVxzPjwRGscsH6Ei7Cgo3EQciSDH99PCNonSP9vTVE, Piece ID: YG22O5IGTGUBMAAODU3T7EBKRLRATKLYBNVSRAZ7APWNIAL2NXKA) context deadline exceeded; piecestore: (Node ID: 15KQ2SjAnXeXBccMHNKZff2PLjeLd2a1WNFtk1ebx1GBdhd3kw, Piece ID: VZWPUBPHQRLPE67H7PPWBO4LGQ7ANOZAFJY3P6VIGQEYJZI5DYYA) context deadline exceeded; piecestore: (Node ID: 12eEQzjo5uQZGL1JR2AyinqCXxderwuqTKvTmoFZ3uNDqwE6WAo, Piece ID: E2SFJCYFLH6UQZ5BZN2YOB3NKEMLRGAJB2GPIXIUBBK53P7VNDOQ) context deadline exceeded; piecestore: (Node ID: 12vQvbGahDxH7tm6NWygYvfuCerQaLqA3BQNEG7bSDKgEKRbUaJ, Piece ID: YTRN7GBUWG5QQIYBU4RB7VK7LS2PTRXTV44BN7P5GCO2VF5F5D2A) context deadline exceeded; piecestore: (Node ID: 12XCy3WvXLNmgSisHzPVy8MFWS8jiA5gnTNTiux7s4FaLnaJWAU, Piece ID: ZD4QR2TJPVJLE545HQGYDR5NDZPEWUMQISU3A2PNTVBE7WH6AU6A) context deadline exceeded Duration: 5.052249267s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.841195102s trace: https://jaeger.dev.storj.io/trace/574375048ac9530f FAIL (2025-12-07 22:47:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.841226151s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 4.890117442s trace: https://jaeger.dev.storj.io/trace/696a333689b50b84 FAIL (2025-12-07 22:46:30 UTC) orbiter warning: operation: sync2.Copy uplink: piecestore: (Node ID: 1ByMj85eQ9yd81bpEjSKrqFTEnLsuV132WoAMBxHrd12jgqddx, Piece ID: PK5SS7JPDNWVGZ4EJ43YWL4MNKGSCS7564EM62UJPBRHXKB4KBHA) context deadline exceeded; piecestore: (Node ID: 1LmXgEBf4pEfYehAES7kR3sVLRt7QxS2pvttiNtqaDnYp7PDzy, Piece ID: Z7PBN4D4PZIWZZZDO7A666MCKEUIZ7ZG3O64BVY3YITRZ7SJISEQ) context deadline exceeded; piecestore: (Node ID: 12Hq4E5wWAPzrxZacP6gbFpKiBvbRXnPdSR4z9jRU443yCFqS2r, Piece ID: CAAZO3SF5Y2YQL2VXBEONDARGL3AXYRZWJSFH3CMANA4J5GOHD7Q) context deadline exceeded; piecestore: (Node ID: 12SQdxBgr7pu6Kz3M16tJjezo5bMiG1gKB57kS3PWN3ibsFVEJ8, Piece ID: 5RB4OO2XHF7WVT7HVTEQL4P4HAJYP3WRHDQVFWERDRBNKUVXYKSA) context deadline exceeded; piecestore: (Node ID: 1hz2v7h419qCvgzkvckp6PYDeQRKRGyHsLtyMGeG1VMa58A8EF, Piece ID: 2YOEIDEDQHBSYGLLEYYHV3BL63VCF2X55JPY5TKSBJSPOKUFEHHQ) context deadline exceeded; piecestore: (Node ID: 1uT7ntK6KAVDSErVdc37UcSd7xzCd3Dont23WitCYrA6KN4h4r, Piece ID: IGM33ZIHTUUKUXREFCY22HT7GRRK6RFODU7SU2N5ZXHWVIRO57KA) context deadline exceeded; piecestore: (Node ID: 1Ea8H6e3FPUzq42DSetCZ9e9ZZWtWpqyDy3fdYBucQDcCci8NN, Piece ID: WAZBXZ323RJVVK3IOKEHLLQJFA5LTNPQV72IP3GS3337GH4VGDAQ) context deadline exceeded; piecestore: (Node ID: 12ifo86KtZXZxEbUAxnvmX5tAg1C27mYub4VUTc9CkWTmpzPP3J, Piece ID: E4XT2PLZFP2HNL62PO5PMMXGQLMBDWV6FP63YSKKOV6LILK4EZLQ) context deadline exceeded; piecestore: (Node ID: 1APgLgqBaSndUajVXyuLWvkM2jbxwuYDkJhxVhweWNDjuju5A3, Piece ID: CDKE76XLSEISNM37HMKFG7STBORFP6NN47F326YW7B6HBEBGSZ4A) context deadline exceeded; piecestore: (Node ID: 12k5wzLZYUYDuQcbMv8NiXV2XBuZvbXbio3GErJYkzNEezwMwLG, Piece ID: TY4UXQXVTIAU2EWYBAFFPNM5OAQHJ5BJL24MGT7V5MSCGV4IDQ2Q) context deadline exceeded; piecestore: (Node ID: 12Ae31qNHzgX2cfrD8EVZqvJ9J95hijL4TEYtuGQPSpWNABG4kw, Piece ID: BIQOY7CGH7SDRD3YZHEBFNBIXUTAUXJMKHUOPTZIFSFXQCIVKSRA) context deadline exceeded; piecestore: (Node ID: 12EaBCW6FX8WY7HGhJJvDZrUy4QuBn8mH7xLz3v4BQA4rdKfKEg, Piece ID: ZJKUDMMW5TXZXRM63BHJ2IAOTQVRXQRMCLQFE6NYRQENBCI7VWIA) context deadline exceeded; piecestore: (Node ID: 12QzLS4YZYRfg6Aw82iFPkAioAuMn22Sf2EkwJFiZZsBDdvv1vS, Piece ID: HG7A6G3H3WOHBUO2QXHZZSYCYRPGZ3NW5MI7VE7F64GJHOHUQ34Q) context deadline exceeded; piecestore: (Node ID: 1vSFRya2YG5Agcz3txzeFfEvtymD4qGEBqrZjFYwEBR3L6tSdU, Piece ID: 5CBLLQNEOPVUGWZ3GYIXS65SDMJESBXYPPUUXH55HVRRKDGDYCPA) context deadline exceeded; piecestore: (Node ID: 12Tw2fxHJaQWpMnX3zZRtmp4Rpy9DEGx6vdZdo6x4eC6nvNgj7K, Piece ID: ULAMA6H4CRAUSN6KYQ2XXVJ4BMCJWB7WWTKBPTSQ7HF2IAZOHWNQ) context deadline exceeded; piecestore: (Node ID: 12aNknfsj5LsTiS8qb42nUSX1dYMhKbwesR6sZJiZtRJWzvfyqk, Piece ID: CKQCDBMNHUROXS34YWSXMJI4Z2A7DQHGKFTJBSGZ463YNGO62ZPA) context deadline exceeded; piecestore: (Node ID: 12UyANaqcy2NBjkmrA1buuHKpJcCxVz8vMpBzqujgmTQB78DY4u, Piece ID: POQJTXSQ45VBTFONHUXVLWMIY62MSQL6VZM6OOAHO42YIIZ6JJCA) context deadline exceeded; piecestore: (Node ID: 12YiXJaHTWUBq5fbKoFHHzeVxdSqpzeMb838cZgynEur4xHAEP, Piece ID: 2NRFXZUZJMBOCUXG6ZF2NRA4ILQ34UUPIPXHZE3QXRMAIWPUQTQA) context deadline exceeded; piecestore: (Node ID: 12uLGjR4eCbBC8GY5vPc4UJmj4AkzGxQd7WQeSo5aw4AxbpaF6J, Piece ID: XGCOLCZIHEWZAJHYAX45DH5LISJCNAWTPHRNM6Q2IZ3UC4TRDDZA) context deadline exceeded; piecestore: (Node ID: 12ntDfA4LBBPkXUioZRWwQhMMA7kBN5rpMd91BgMi7tw2uuRi7n, Piece ID: 4D7CGCOOSMD6WHPV6PRLKDAME6A6SKX3NT32XS2KLYZKO2CJTURA) context deadline exceeded; piecestore: (Node ID: 12QjxocGH8nPtfBmZqTVmV3sssQs5nR8QsWaVga3BdKU97XgPBz, Piece ID: O75ISG3WWM46CC4ZALCC56AAVRXMRPYUMUOMWZVIEWNTLQ5B6UVQ) context deadline exceeded; piecestore: (Node ID: 1qEoNfbFCs5TobVBDG2aV1qSNU6At8ZUxSeGGaYFytgCcArXg7, Piece ID: VNDX43FYBGVJMPQ4773RC5OSYOPTJTYD26RI37ULIRNZKKLATSZA) context deadline exceeded; piecestore: (Node ID: 12hRf4FXCVVxzPjwRGscsH6Ei7Cgo3EQciSDH99PCNonSP9vTVE, Piece ID: YG22O5IGTGUBMAAODU3T7EBKRLRATKLYBNVSRAZ7APWNIAL2NXKA) context deadline exceeded; piecestore: (Node ID: 15KQ2SjAnXeXBccMHNKZff2PLjeLd2a1WNFtk1ebx1GBdhd3kw, Piece ID: VZWPUBPHQRLPE67H7PPWBO4LGQ7ANOZAFJY3P6VIGQEYJZI5DYYA) context deadline exceeded; piecestore: (Node ID: 12eEQzjo5uQZGL1JR2AyinqCXxderwuqTKvTmoFZ3uNDqwE6WAo, Piece ID: E2SFJCYFLH6UQZ5BZN2YOB3NKEMLRGAJB2GPIXIUBBK53P7VNDOQ) context deadline exceeded; piecestore: (Node ID: 12vQvbGahDxH7tm6NWygYvfuCerQaLqA3BQNEG7bSDKgEKRbUaJ, Piece ID: YTRN7GBUWG5QQIYBU4RB7VK7LS2PTRXTV44BN7P5GCO2VF5F5D2A) context deadline exceeded; piecestore: (Node ID: 12XCy3WvXLNmgSisHzPVy8MFWS8jiA5gnTNTiux7s4FaLnaJWAU, Piece ID: ZD4QR2TJPVJLE545HQGYDR5NDZPEWUMQISU3A2PNTVBE7WH6AU6A) context deadline exceeded Duration: 4.890232261s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.798045547s trace: https://jaeger.dev.storj.io/trace/574375048ac9530f FAIL (2025-12-07 22:46:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.798075212s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.968079094s trace: https://jaeger.dev.storj.io/trace/574375048ac9530f FAIL (2025-12-07 22:45:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.968102379s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 4.933593438s trace: https://jaeger.dev.storj.io/trace/696a333689b50b84 FAIL (2025-12-07 22:45:28 UTC) orbiter warning: operation: sync2.Copy uplink: piecestore: (Node ID: 1ByMj85eQ9yd81bpEjSKrqFTEnLsuV132WoAMBxHrd12jgqddx, Piece ID: PK5SS7JPDNWVGZ4EJ43YWL4MNKGSCS7564EM62UJPBRHXKB4KBHA) context deadline exceeded; piecestore: (Node ID: 1LmXgEBf4pEfYehAES7kR3sVLRt7QxS2pvttiNtqaDnYp7PDzy, Piece ID: Z7PBN4D4PZIWZZZDO7A666MCKEUIZ7ZG3O64BVY3YITRZ7SJISEQ) context deadline exceeded; piecestore: (Node ID: 12Hq4E5wWAPzrxZacP6gbFpKiBvbRXnPdSR4z9jRU443yCFqS2r, Piece ID: CAAZO3SF5Y2YQL2VXBEONDARGL3AXYRZWJSFH3CMANA4J5GOHD7Q) context deadline exceeded; piecestore: (Node ID: 12SQdxBgr7pu6Kz3M16tJjezo5bMiG1gKB57kS3PWN3ibsFVEJ8, Piece ID: 5RB4OO2XHF7WVT7HVTEQL4P4HAJYP3WRHDQVFWERDRBNKUVXYKSA) context deadline exceeded; piecestore: (Node ID: 1hz2v7h419qCvgzkvckp6PYDeQRKRGyHsLtyMGeG1VMa58A8EF, Piece ID: 2YOEIDEDQHBSYGLLEYYHV3BL63VCF2X55JPY5TKSBJSPOKUFEHHQ) context deadline exceeded; piecestore: (Node ID: 1uT7ntK6KAVDSErVdc37UcSd7xzCd3Dont23WitCYrA6KN4h4r, Piece ID: IGM33ZIHTUUKUXREFCY22HT7GRRK6RFODU7SU2N5ZXHWVIRO57KA) context deadline exceeded; piecestore: (Node ID: 1Ea8H6e3FPUzq42DSetCZ9e9ZZWtWpqyDy3fdYBucQDcCci8NN, Piece ID: WAZBXZ323RJVVK3IOKEHLLQJFA5LTNPQV72IP3GS3337GH4VGDAQ) context deadline exceeded; piecestore: (Node ID: 12ifo86KtZXZxEbUAxnvmX5tAg1C27mYub4VUTc9CkWTmpzPP3J, Piece ID: E4XT2PLZFP2HNL62PO5PMMXGQLMBDWV6FP63YSKKOV6LILK4EZLQ) context deadline exceeded; piecestore: (Node ID: 1APgLgqBaSndUajVXyuLWvkM2jbxwuYDkJhxVhweWNDjuju5A3, Piece ID: CDKE76XLSEISNM37HMKFG7STBORFP6NN47F326YW7B6HBEBGSZ4A) context deadline exceeded; piecestore: (Node ID: 12k5wzLZYUYDuQcbMv8NiXV2XBuZvbXbio3GErJYkzNEezwMwLG, Piece ID: TY4UXQXVTIAU2EWYBAFFPNM5OAQHJ5BJL24MGT7V5MSCGV4IDQ2Q) context deadline exceeded; piecestore: (Node ID: 12Ae31qNHzgX2cfrD8EVZqvJ9J95hijL4TEYtuGQPSpWNABG4kw, Piece ID: BIQOY7CGH7SDRD3YZHEBFNBIXUTAUXJMKHUOPTZIFSFXQCIVKSRA) context deadline exceeded; piecestore: (Node ID: 12EaBCW6FX8WY7HGhJJvDZrUy4QuBn8mH7xLz3v4BQA4rdKfKEg, Piece ID: ZJKUDMMW5TXZXRM63BHJ2IAOTQVRXQRMCLQFE6NYRQENBCI7VWIA) context deadline exceeded; piecestore: (Node ID: 12QzLS4YZYRfg6Aw82iFPkAioAuMn22Sf2EkwJFiZZsBDdvv1vS, Piece ID: HG7A6G3H3WOHBUO2QXHZZSYCYRPGZ3NW5MI7VE7F64GJHOHUQ34Q) context deadline exceeded; piecestore: (Node ID: 1vSFRya2YG5Agcz3txzeFfEvtymD4qGEBqrZjFYwEBR3L6tSdU, Piece ID: 5CBLLQNEOPVUGWZ3GYIXS65SDMJESBXYPPUUXH55HVRRKDGDYCPA) context deadline exceeded; piecestore: (Node ID: 12Tw2fxHJaQWpMnX3zZRtmp4Rpy9DEGx6vdZdo6x4eC6nvNgj7K, Piece ID: ULAMA6H4CRAUSN6KYQ2XXVJ4BMCJWB7WWTKBPTSQ7HF2IAZOHWNQ) context deadline exceeded; piecestore: (Node ID: 12aNknfsj5LsTiS8qb42nUSX1dYMhKbwesR6sZJiZtRJWzvfyqk, Piece ID: CKQCDBMNHUROXS34YWSXMJI4Z2A7DQHGKFTJBSGZ463YNGO62ZPA) context deadline exceeded; piecestore: (Node ID: 12UyANaqcy2NBjkmrA1buuHKpJcCxVz8vMpBzqujgmTQB78DY4u, Piece ID: POQJTXSQ45VBTFONHUXVLWMIY62MSQL6VZM6OOAHO42YIIZ6JJCA) context deadline exceeded; piecestore: (Node ID: 12YiXJaHTWUBq5fbKoFHHzeVxdSqpzeMb838cZgynEur4xHAEP, Piece ID: 2NRFXZUZJMBOCUXG6ZF2NRA4ILQ34UUPIPXHZE3QXRMAIWPUQTQA) context deadline exceeded; piecestore: (Node ID: 12uLGjR4eCbBC8GY5vPc4UJmj4AkzGxQd7WQeSo5aw4AxbpaF6J, Piece ID: XGCOLCZIHEWZAJHYAX45DH5LISJCNAWTPHRNM6Q2IZ3UC4TRDDZA) context deadline exceeded; piecestore: (Node ID: 12ntDfA4LBBPkXUioZRWwQhMMA7kBN5rpMd91BgMi7tw2uuRi7n, Piece ID: 4D7CGCOOSMD6WHPV6PRLKDAME6A6SKX3NT32XS2KLYZKO2CJTURA) context deadline exceeded; piecestore: (Node ID: 12QjxocGH8nPtfBmZqTVmV3sssQs5nR8QsWaVga3BdKU97XgPBz, Piece ID: O75ISG3WWM46CC4ZALCC56AAVRXMRPYUMUOMWZVIEWNTLQ5B6UVQ) context deadline exceeded; piecestore: (Node ID: 1qEoNfbFCs5TobVBDG2aV1qSNU6At8ZUxSeGGaYFytgCcArXg7, Piece ID: VNDX43FYBGVJMPQ4773RC5OSYOPTJTYD26RI37ULIRNZKKLATSZA) context deadline exceeded; piecestore: (Node ID: 12hRf4FXCVVxzPjwRGscsH6Ei7Cgo3EQciSDH99PCNonSP9vTVE, Piece ID: YG22O5IGTGUBMAAODU3T7EBKRLRATKLYBNVSRAZ7APWNIAL2NXKA) context deadline exceeded; piecestore: (Node ID: 15KQ2SjAnXeXBccMHNKZff2PLjeLd2a1WNFtk1ebx1GBdhd3kw, Piece ID: VZWPUBPHQRLPE67H7PPWBO4LGQ7ANOZAFJY3P6VIGQEYJZI5DYYA) context deadline exceeded; piecestore: (Node ID: 12eEQzjo5uQZGL1JR2AyinqCXxderwuqTKvTmoFZ3uNDqwE6WAo, Piece ID: E2SFJCYFLH6UQZ5BZN2YOB3NKEMLRGAJB2GPIXIUBBK53P7VNDOQ) context deadline exceeded; piecestore: (Node ID: 12vQvbGahDxH7tm6NWygYvfuCerQaLqA3BQNEG7bSDKgEKRbUaJ, Piece ID: YTRN7GBUWG5QQIYBU4RB7VK7LS2PTRXTV44BN7P5GCO2VF5F5D2A) context deadline exceeded; piecestore: (Node ID: 12XCy3WvXLNmgSisHzPVy8MFWS8jiA5gnTNTiux7s4FaLnaJWAU, Piece ID: ZD4QR2TJPVJLE545HQGYDR5NDZPEWUMQISU3A2PNTVBE7WH6AU6A) context deadline exceeded Duration: 4.933622542s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.882453312s trace: https://jaeger.dev.storj.io/trace/574375048ac9530f FAIL (2025-12-07 22:45:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.882512104s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.001258618s trace: https://jaeger.dev.storj.io/trace/574375048ac9530f FAIL (2025-12-07 22:45:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001278476s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 4.974521501s trace: https://jaeger.dev.storj.io/trace/696a333689b50b84 FAIL (2025-12-07 22:44:29 UTC) orbiter warning: operation: sync2.Copy uplink: piecestore: (Node ID: 1ByMj85eQ9yd81bpEjSKrqFTEnLsuV132WoAMBxHrd12jgqddx, Piece ID: PK5SS7JPDNWVGZ4EJ43YWL4MNKGSCS7564EM62UJPBRHXKB4KBHA) context deadline exceeded; piecestore: (Node ID: 1LmXgEBf4pEfYehAES7kR3sVLRt7QxS2pvttiNtqaDnYp7PDzy, Piece ID: Z7PBN4D4PZIWZZZDO7A666MCKEUIZ7ZG3O64BVY3YITRZ7SJISEQ) context deadline exceeded; piecestore: (Node ID: 12Hq4E5wWAPzrxZacP6gbFpKiBvbRXnPdSR4z9jRU443yCFqS2r, Piece ID: CAAZO3SF5Y2YQL2VXBEONDARGL3AXYRZWJSFH3CMANA4J5GOHD7Q) context deadline exceeded; piecestore: (Node ID: 12SQdxBgr7pu6Kz3M16tJjezo5bMiG1gKB57kS3PWN3ibsFVEJ8, Piece ID: 5RB4OO2XHF7WVT7HVTEQL4P4HAJYP3WRHDQVFWERDRBNKUVXYKSA) context deadline exceeded; piecestore: (Node ID: 1hz2v7h419qCvgzkvckp6PYDeQRKRGyHsLtyMGeG1VMa58A8EF, Piece ID: 2YOEIDEDQHBSYGLLEYYHV3BL63VCF2X55JPY5TKSBJSPOKUFEHHQ) context deadline exceeded; piecestore: (Node ID: 1uT7ntK6KAVDSErVdc37UcSd7xzCd3Dont23WitCYrA6KN4h4r, Piece ID: IGM33ZIHTUUKUXREFCY22HT7GRRK6RFODU7SU2N5ZXHWVIRO57KA) context deadline exceeded; piecestore: (Node ID: 1Ea8H6e3FPUzq42DSetCZ9e9ZZWtWpqyDy3fdYBucQDcCci8NN, Piece ID: WAZBXZ323RJVVK3IOKEHLLQJFA5LTNPQV72IP3GS3337GH4VGDAQ) context deadline exceeded; piecestore: (Node ID: 12ifo86KtZXZxEbUAxnvmX5tAg1C27mYub4VUTc9CkWTmpzPP3J, Piece ID: E4XT2PLZFP2HNL62PO5PMMXGQLMBDWV6FP63YSKKOV6LILK4EZLQ) context deadline exceeded; piecestore: (Node ID: 1APgLgqBaSndUajVXyuLWvkM2jbxwuYDkJhxVhweWNDjuju5A3, Piece ID: CDKE76XLSEISNM37HMKFG7STBORFP6NN47F326YW7B6HBEBGSZ4A) context deadline exceeded; piecestore: (Node ID: 12k5wzLZYUYDuQcbMv8NiXV2XBuZvbXbio3GErJYkzNEezwMwLG, Piece ID: TY4UXQXVTIAU2EWYBAFFPNM5OAQHJ5BJL24MGT7V5MSCGV4IDQ2Q) context deadline exceeded; piecestore: (Node ID: 12Ae31qNHzgX2cfrD8EVZqvJ9J95hijL4TEYtuGQPSpWNABG4kw, Piece ID: BIQOY7CGH7SDRD3YZHEBFNBIXUTAUXJMKHUOPTZIFSFXQCIVKSRA) context deadline exceeded; piecestore: (Node ID: 12EaBCW6FX8WY7HGhJJvDZrUy4QuBn8mH7xLz3v4BQA4rdKfKEg, Piece ID: ZJKUDMMW5TXZXRM63BHJ2IAOTQVRXQRMCLQFE6NYRQENBCI7VWIA) context deadline exceeded; piecestore: (Node ID: 12QzLS4YZYRfg6Aw82iFPkAioAuMn22Sf2EkwJFiZZsBDdvv1vS, Piece ID: HG7A6G3H3WOHBUO2QXHZZSYCYRPGZ3NW5MI7VE7F64GJHOHUQ34Q) context deadline exceeded; piecestore: (Node ID: 1vSFRya2YG5Agcz3txzeFfEvtymD4qGEBqrZjFYwEBR3L6tSdU, Piece ID: 5CBLLQNEOPVUGWZ3GYIXS65SDMJESBXYPPUUXH55HVRRKDGDYCPA) context deadline exceeded; piecestore: (Node ID: 12Tw2fxHJaQWpMnX3zZRtmp4Rpy9DEGx6vdZdo6x4eC6nvNgj7K, Piece ID: ULAMA6H4CRAUSN6KYQ2XXVJ4BMCJWB7WWTKBPTSQ7HF2IAZOHWNQ) context deadline exceeded; piecestore: (Node ID: 12aNknfsj5LsTiS8qb42nUSX1dYMhKbwesR6sZJiZtRJWzvfyqk, Piece ID: CKQCDBMNHUROXS34YWSXMJI4Z2A7DQHGKFTJBSGZ463YNGO62ZPA) context deadline exceeded; piecestore: (Node ID: 12UyANaqcy2NBjkmrA1buuHKpJcCxVz8vMpBzqujgmTQB78DY4u, Piece ID: POQJTXSQ45VBTFONHUXVLWMIY62MSQL6VZM6OOAHO42YIIZ6JJCA) context deadline exceeded; piecestore: (Node ID: 12YiXJaHTWUBq5fbKoFHHzeVxdSqpzeMb838cZgynEur4xHAEP, Piece ID: 2NRFXZUZJMBOCUXG6ZF2NRA4ILQ34UUPIPXHZE3QXRMAIWPUQTQA) context deadline exceeded; piecestore: (Node ID: 12uLGjR4eCbBC8GY5vPc4UJmj4AkzGxQd7WQeSo5aw4AxbpaF6J, Piece ID: XGCOLCZIHEWZAJHYAX45DH5LISJCNAWTPHRNM6Q2IZ3UC4TRDDZA) context deadline exceeded; piecestore: (Node ID: 12ntDfA4LBBPkXUioZRWwQhMMA7kBN5rpMd91BgMi7tw2uuRi7n, Piece ID: 4D7CGCOOSMD6WHPV6PRLKDAME6A6SKX3NT32XS2KLYZKO2CJTURA) context deadline exceeded; piecestore: (Node ID: 12QjxocGH8nPtfBmZqTVmV3sssQs5nR8QsWaVga3BdKU97XgPBz, Piece ID: O75ISG3WWM46CC4ZALCC56AAVRXMRPYUMUOMWZVIEWNTLQ5B6UVQ) context deadline exceeded; piecestore: (Node ID: 1qEoNfbFCs5TobVBDG2aV1qSNU6At8ZUxSeGGaYFytgCcArXg7, Piece ID: VNDX43FYBGVJMPQ4773RC5OSYOPTJTYD26RI37ULIRNZKKLATSZA) context deadline exceeded; piecestore: (Node ID: 12hRf4FXCVVxzPjwRGscsH6Ei7Cgo3EQciSDH99PCNonSP9vTVE, Piece ID: YG22O5IGTGUBMAAODU3T7EBKRLRATKLYBNVSRAZ7APWNIAL2NXKA) context deadline exceeded; piecestore: (Node ID: 15KQ2SjAnXeXBccMHNKZff2PLjeLd2a1WNFtk1ebx1GBdhd3kw, Piece ID: VZWPUBPHQRLPE67H7PPWBO4LGQ7ANOZAFJY3P6VIGQEYJZI5DYYA) context deadline exceeded; piecestore: (Node ID: 12eEQzjo5uQZGL1JR2AyinqCXxderwuqTKvTmoFZ3uNDqwE6WAo, Piece ID: E2SFJCYFLH6UQZ5BZN2YOB3NKEMLRGAJB2GPIXIUBBK53P7VNDOQ) context deadline exceeded; piecestore: (Node ID: 12vQvbGahDxH7tm6NWygYvfuCerQaLqA3BQNEG7bSDKgEKRbUaJ, Piece ID: YTRN7GBUWG5QQIYBU4RB7VK7LS2PTRXTV44BN7P5GCO2VF5F5D2A) context deadline exceeded; piecestore: (Node ID: 12XCy3WvXLNmgSisHzPVy8MFWS8jiA5gnTNTiux7s4FaLnaJWAU, Piece ID: ZD4QR2TJPVJLE545HQGYDR5NDZPEWUMQISU3A2PNTVBE7WH6AU6A) context deadline exceeded Duration: 4.974549836s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 4.903109449s trace: https://jaeger.dev.storj.io/trace/696a333689b50b84 FAIL (2025-12-07 22:43:32 UTC) orbiter warning: operation: sync2.Copy uplink: piecestore: (Node ID: 1ByMj85eQ9yd81bpEjSKrqFTEnLsuV132WoAMBxHrd12jgqddx, Piece ID: PK5SS7JPDNWVGZ4EJ43YWL4MNKGSCS7564EM62UJPBRHXKB4KBHA) context deadline exceeded; piecestore: (Node ID: 1LmXgEBf4pEfYehAES7kR3sVLRt7QxS2pvttiNtqaDnYp7PDzy, Piece ID: Z7PBN4D4PZIWZZZDO7A666MCKEUIZ7ZG3O64BVY3YITRZ7SJISEQ) context deadline exceeded; piecestore: (Node ID: 12Hq4E5wWAPzrxZacP6gbFpKiBvbRXnPdSR4z9jRU443yCFqS2r, Piece ID: CAAZO3SF5Y2YQL2VXBEONDARGL3AXYRZWJSFH3CMANA4J5GOHD7Q) context deadline exceeded; piecestore: (Node ID: 12SQdxBgr7pu6Kz3M16tJjezo5bMiG1gKB57kS3PWN3ibsFVEJ8, Piece ID: 5RB4OO2XHF7WVT7HVTEQL4P4HAJYP3WRHDQVFWERDRBNKUVXYKSA) context deadline exceeded; piecestore: (Node ID: 1hz2v7h419qCvgzkvckp6PYDeQRKRGyHsLtyMGeG1VMa58A8EF, Piece ID: 2YOEIDEDQHBSYGLLEYYHV3BL63VCF2X55JPY5TKSBJSPOKUFEHHQ) context deadline exceeded; piecestore: (Node ID: 1uT7ntK6KAVDSErVdc37UcSd7xzCd3Dont23WitCYrA6KN4h4r, Piece ID: IGM33ZIHTUUKUXREFCY22HT7GRRK6RFODU7SU2N5ZXHWVIRO57KA) context deadline exceeded; piecestore: (Node ID: 1Ea8H6e3FPUzq42DSetCZ9e9ZZWtWpqyDy3fdYBucQDcCci8NN, Piece ID: WAZBXZ323RJVVK3IOKEHLLQJFA5LTNPQV72IP3GS3337GH4VGDAQ) context deadline exceeded; piecestore: (Node ID: 12ifo86KtZXZxEbUAxnvmX5tAg1C27mYub4VUTc9CkWTmpzPP3J, Piece ID: E4XT2PLZFP2HNL62PO5PMMXGQLMBDWV6FP63YSKKOV6LILK4EZLQ) context deadline exceeded; piecestore: (Node ID: 1APgLgqBaSndUajVXyuLWvkM2jbxwuYDkJhxVhweWNDjuju5A3, Piece ID: CDKE76XLSEISNM37HMKFG7STBORFP6NN47F326YW7B6HBEBGSZ4A) context deadline exceeded; piecestore: (Node ID: 12k5wzLZYUYDuQcbMv8NiXV2XBuZvbXbio3GErJYkzNEezwMwLG, Piece ID: TY4UXQXVTIAU2EWYBAFFPNM5OAQHJ5BJL24MGT7V5MSCGV4IDQ2Q) context deadline exceeded; piecestore: (Node ID: 12Ae31qNHzgX2cfrD8EVZqvJ9J95hijL4TEYtuGQPSpWNABG4kw, Piece ID: BIQOY7CGH7SDRD3YZHEBFNBIXUTAUXJMKHUOPTZIFSFXQCIVKSRA) context deadline exceeded; piecestore: (Node ID: 12EaBCW6FX8WY7HGhJJvDZrUy4QuBn8mH7xLz3v4BQA4rdKfKEg, Piece ID: ZJKUDMMW5TXZXRM63BHJ2IAOTQVRXQRMCLQFE6NYRQENBCI7VWIA) context deadline exceeded; piecestore: (Node ID: 12QzLS4YZYRfg6Aw82iFPkAioAuMn22Sf2EkwJFiZZsBDdvv1vS, Piece ID: HG7A6G3H3WOHBUO2QXHZZSYCYRPGZ3NW5MI7VE7F64GJHOHUQ34Q) context deadline exceeded; piecestore: (Node ID: 1vSFRya2YG5Agcz3txzeFfEvtymD4qGEBqrZjFYwEBR3L6tSdU, Piece ID: 5CBLLQNEOPVUGWZ3GYIXS65SDMJESBXYPPUUXH55HVRRKDGDYCPA) context deadline exceeded; piecestore: (Node ID: 12Tw2fxHJaQWpMnX3zZRtmp4Rpy9DEGx6vdZdo6x4eC6nvNgj7K, Piece ID: ULAMA6H4CRAUSN6KYQ2XXVJ4BMCJWB7WWTKBPTSQ7HF2IAZOHWNQ) context deadline exceeded; piecestore: (Node ID: 12aNknfsj5LsTiS8qb42nUSX1dYMhKbwesR6sZJiZtRJWzvfyqk, Piece ID: CKQCDBMNHUROXS34YWSXMJI4Z2A7DQHGKFTJBSGZ463YNGO62ZPA) context deadline exceeded; piecestore: (Node ID: 12UyANaqcy2NBjkmrA1buuHKpJcCxVz8vMpBzqujgmTQB78DY4u, Piece ID: POQJTXSQ45VBTFONHUXVLWMIY62MSQL6VZM6OOAHO42YIIZ6JJCA) context deadline exceeded; piecestore: (Node ID: 12YiXJaHTWUBq5fbKoFHHzeVxdSqpzeMb838cZgynEur4xHAEP, Piece ID: 2NRFXZUZJMBOCUXG6ZF2NRA4ILQ34UUPIPXHZE3QXRMAIWPUQTQA) context deadline exceeded; piecestore: (Node ID: 12uLGjR4eCbBC8GY5vPc4UJmj4AkzGxQd7WQeSo5aw4AxbpaF6J, Piece ID: XGCOLCZIHEWZAJHYAX45DH5LISJCNAWTPHRNM6Q2IZ3UC4TRDDZA) context deadline exceeded; piecestore: (Node ID: 12ntDfA4LBBPkXUioZRWwQhMMA7kBN5rpMd91BgMi7tw2uuRi7n, Piece ID: 4D7CGCOOSMD6WHPV6PRLKDAME6A6SKX3NT32XS2KLYZKO2CJTURA) context deadline exceeded; piecestore: (Node ID: 12QjxocGH8nPtfBmZqTVmV3sssQs5nR8QsWaVga3BdKU97XgPBz, Piece ID: O75ISG3WWM46CC4ZALCC56AAVRXMRPYUMUOMWZVIEWNTLQ5B6UVQ) context deadline exceeded; piecestore: (Node ID: 1qEoNfbFCs5TobVBDG2aV1qSNU6At8ZUxSeGGaYFytgCcArXg7, Piece ID: VNDX43FYBGVJMPQ4773RC5OSYOPTJTYD26RI37ULIRNZKKLATSZA) context deadline exceeded; piecestore: (Node ID: 12hRf4FXCVVxzPjwRGscsH6Ei7Cgo3EQciSDH99PCNonSP9vTVE, Piece ID: YG22O5IGTGUBMAAODU3T7EBKRLRATKLYBNVSRAZ7APWNIAL2NXKA) context deadline exceeded; piecestore: (Node ID: 15KQ2SjAnXeXBccMHNKZff2PLjeLd2a1WNFtk1ebx1GBdhd3kw, Piece ID: VZWPUBPHQRLPE67H7PPWBO4LGQ7ANOZAFJY3P6VIGQEYJZI5DYYA) context deadline exceeded; piecestore: (Node ID: 12eEQzjo5uQZGL1JR2AyinqCXxderwuqTKvTmoFZ3uNDqwE6WAo, Piece ID: E2SFJCYFLH6UQZ5BZN2YOB3NKEMLRGAJB2GPIXIUBBK53P7VNDOQ) context deadline exceeded; piecestore: (Node ID: 12vQvbGahDxH7tm6NWygYvfuCerQaLqA3BQNEG7bSDKgEKRbUaJ, Piece ID: YTRN7GBUWG5QQIYBU4RB7VK7LS2PTRXTV44BN7P5GCO2VF5F5D2A) context deadline exceeded; piecestore: (Node ID: 12XCy3WvXLNmgSisHzPVy8MFWS8jiA5gnTNTiux7s4FaLnaJWAU, Piece ID: ZD4QR2TJPVJLE545HQGYDR5NDZPEWUMQISU3A2PNTVBE7WH6AU6A) context deadline exceeded Duration: 4.903139406s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 4.845905323s trace: https://jaeger.dev.storj.io/trace/696a333689b50b84 FAIL (2025-12-07 22:42:32 UTC) orbiter warning: operation: sync2.Copy uplink: piecestore: (Node ID: 1ByMj85eQ9yd81bpEjSKrqFTEnLsuV132WoAMBxHrd12jgqddx, Piece ID: PK5SS7JPDNWVGZ4EJ43YWL4MNKGSCS7564EM62UJPBRHXKB4KBHA) context deadline exceeded; piecestore: (Node ID: 1LmXgEBf4pEfYehAES7kR3sVLRt7QxS2pvttiNtqaDnYp7PDzy, Piece ID: Z7PBN4D4PZIWZZZDO7A666MCKEUIZ7ZG3O64BVY3YITRZ7SJISEQ) context deadline exceeded; piecestore: (Node ID: 12Hq4E5wWAPzrxZacP6gbFpKiBvbRXnPdSR4z9jRU443yCFqS2r, Piece ID: CAAZO3SF5Y2YQL2VXBEONDARGL3AXYRZWJSFH3CMANA4J5GOHD7Q) context deadline exceeded; piecestore: (Node ID: 12SQdxBgr7pu6Kz3M16tJjezo5bMiG1gKB57kS3PWN3ibsFVEJ8, Piece ID: 5RB4OO2XHF7WVT7HVTEQL4P4HAJYP3WRHDQVFWERDRBNKUVXYKSA) context deadline exceeded; piecestore: (Node ID: 1hz2v7h419qCvgzkvckp6PYDeQRKRGyHsLtyMGeG1VMa58A8EF, Piece ID: 2YOEIDEDQHBSYGLLEYYHV3BL63VCF2X55JPY5TKSBJSPOKUFEHHQ) context deadline exceeded; piecestore: (Node ID: 1uT7ntK6KAVDSErVdc37UcSd7xzCd3Dont23WitCYrA6KN4h4r, Piece ID: IGM33ZIHTUUKUXREFCY22HT7GRRK6RFODU7SU2N5ZXHWVIRO57KA) context deadline exceeded; piecestore: (Node ID: 1Ea8H6e3FPUzq42DSetCZ9e9ZZWtWpqyDy3fdYBucQDcCci8NN, Piece ID: WAZBXZ323RJVVK3IOKEHLLQJFA5LTNPQV72IP3GS3337GH4VGDAQ) context deadline exceeded; piecestore: (Node ID: 12ifo86KtZXZxEbUAxnvmX5tAg1C27mYub4VUTc9CkWTmpzPP3J, Piece ID: E4XT2PLZFP2HNL62PO5PMMXGQLMBDWV6FP63YSKKOV6LILK4EZLQ) context deadline exceeded; piecestore: (Node ID: 1APgLgqBaSndUajVXyuLWvkM2jbxwuYDkJhxVhweWNDjuju5A3, Piece ID: CDKE76XLSEISNM37HMKFG7STBORFP6NN47F326YW7B6HBEBGSZ4A) context deadline exceeded; piecestore: (Node ID: 12k5wzLZYUYDuQcbMv8NiXV2XBuZvbXbio3GErJYkzNEezwMwLG, Piece ID: TY4UXQXVTIAU2EWYBAFFPNM5OAQHJ5BJL24MGT7V5MSCGV4IDQ2Q) context deadline exceeded; piecestore: (Node ID: 12Ae31qNHzgX2cfrD8EVZqvJ9J95hijL4TEYtuGQPSpWNABG4kw, Piece ID: BIQOY7CGH7SDRD3YZHEBFNBIXUTAUXJMKHUOPTZIFSFXQCIVKSRA) context deadline exceeded; piecestore: (Node ID: 12EaBCW6FX8WY7HGhJJvDZrUy4QuBn8mH7xLz3v4BQA4rdKfKEg, Piece ID: ZJKUDMMW5TXZXRM63BHJ2IAOTQVRXQRMCLQFE6NYRQENBCI7VWIA) context deadline exceeded; piecestore: (Node ID: 12QzLS4YZYRfg6Aw82iFPkAioAuMn22Sf2EkwJFiZZsBDdvv1vS, Piece ID: HG7A6G3H3WOHBUO2QXHZZSYCYRPGZ3NW5MI7VE7F64GJHOHUQ34Q) context deadline exceeded; piecestore: (Node ID: 1vSFRya2YG5Agcz3txzeFfEvtymD4qGEBqrZjFYwEBR3L6tSdU, Piece ID: 5CBLLQNEOPVUGWZ3GYIXS65SDMJESBXYPPUUXH55HVRRKDGDYCPA) context deadline exceeded; piecestore: (Node ID: 12Tw2fxHJaQWpMnX3zZRtmp4Rpy9DEGx6vdZdo6x4eC6nvNgj7K, Piece ID: ULAMA6H4CRAUSN6KYQ2XXVJ4BMCJWB7WWTKBPTSQ7HF2IAZOHWNQ) context deadline exceeded; piecestore: (Node ID: 12aNknfsj5LsTiS8qb42nUSX1dYMhKbwesR6sZJiZtRJWzvfyqk, Piece ID: CKQCDBMNHUROXS34YWSXMJI4Z2A7DQHGKFTJBSGZ463YNGO62ZPA) context deadline exceeded; piecestore: (Node ID: 12UyANaqcy2NBjkmrA1buuHKpJcCxVz8vMpBzqujgmTQB78DY4u, Piece ID: POQJTXSQ45VBTFONHUXVLWMIY62MSQL6VZM6OOAHO42YIIZ6JJCA) context deadline exceeded; piecestore: (Node ID: 12YiXJaHTWUBq5fbKoFHHzeVxdSqpzeMb838cZgynEur4xHAEP, Piece ID: 2NRFXZUZJMBOCUXG6ZF2NRA4ILQ34UUPIPXHZE3QXRMAIWPUQTQA) context deadline exceeded; piecestore: (Node ID: 12uLGjR4eCbBC8GY5vPc4UJmj4AkzGxQd7WQeSo5aw4AxbpaF6J, Piece ID: XGCOLCZIHEWZAJHYAX45DH5LISJCNAWTPHRNM6Q2IZ3UC4TRDDZA) context deadline exceeded; piecestore: (Node ID: 12ntDfA4LBBPkXUioZRWwQhMMA7kBN5rpMd91BgMi7tw2uuRi7n, Piece ID: 4D7CGCOOSMD6WHPV6PRLKDAME6A6SKX3NT32XS2KLYZKO2CJTURA) context deadline exceeded; piecestore: (Node ID: 12QjxocGH8nPtfBmZqTVmV3sssQs5nR8QsWaVga3BdKU97XgPBz, Piece ID: O75ISG3WWM46CC4ZALCC56AAVRXMRPYUMUOMWZVIEWNTLQ5B6UVQ) context deadline exceeded; piecestore: (Node ID: 1qEoNfbFCs5TobVBDG2aV1qSNU6At8ZUxSeGGaYFytgCcArXg7, Piece ID: VNDX43FYBGVJMPQ4773RC5OSYOPTJTYD26RI37ULIRNZKKLATSZA) context deadline exceeded; piecestore: (Node ID: 12hRf4FXCVVxzPjwRGscsH6Ei7Cgo3EQciSDH99PCNonSP9vTVE, Piece ID: YG22O5IGTGUBMAAODU3T7EBKRLRATKLYBNVSRAZ7APWNIAL2NXKA) context deadline exceeded; piecestore: (Node ID: 15KQ2SjAnXeXBccMHNKZff2PLjeLd2a1WNFtk1ebx1GBdhd3kw, Piece ID: VZWPUBPHQRLPE67H7PPWBO4LGQ7ANOZAFJY3P6VIGQEYJZI5DYYA) context deadline exceeded; piecestore: (Node ID: 12eEQzjo5uQZGL1JR2AyinqCXxderwuqTKvTmoFZ3uNDqwE6WAo, Piece ID: E2SFJCYFLH6UQZ5BZN2YOB3NKEMLRGAJB2GPIXIUBBK53P7VNDOQ) context deadline exceeded; piecestore: (Node ID: 12vQvbGahDxH7tm6NWygYvfuCerQaLqA3BQNEG7bSDKgEKRbUaJ, Piece ID: YTRN7GBUWG5QQIYBU4RB7VK7LS2PTRXTV44BN7P5GCO2VF5F5D2A) context deadline exceeded; piecestore: (Node ID: 12XCy3WvXLNmgSisHzPVy8MFWS8jiA5gnTNTiux7s4FaLnaJWAU, Piece ID: ZD4QR2TJPVJLE545HQGYDR5NDZPEWUMQISU3A2PNTVBE7WH6AU6A) context deadline exceeded Duration: 4.845926795s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 4.958652986s trace: https://jaeger.dev.storj.io/trace/696a333689b50b84 FAIL (2025-12-07 22:41:42 UTC) orbiter warning: operation: sync2.Copy uplink: piecestore: (Node ID: 1ByMj85eQ9yd81bpEjSKrqFTEnLsuV132WoAMBxHrd12jgqddx, Piece ID: PK5SS7JPDNWVGZ4EJ43YWL4MNKGSCS7564EM62UJPBRHXKB4KBHA) context deadline exceeded; piecestore: (Node ID: 1LmXgEBf4pEfYehAES7kR3sVLRt7QxS2pvttiNtqaDnYp7PDzy, Piece ID: Z7PBN4D4PZIWZZZDO7A666MCKEUIZ7ZG3O64BVY3YITRZ7SJISEQ) context deadline exceeded; piecestore: (Node ID: 12Hq4E5wWAPzrxZacP6gbFpKiBvbRXnPdSR4z9jRU443yCFqS2r, Piece ID: CAAZO3SF5Y2YQL2VXBEONDARGL3AXYRZWJSFH3CMANA4J5GOHD7Q) context deadline exceeded; piecestore: (Node ID: 12SQdxBgr7pu6Kz3M16tJjezo5bMiG1gKB57kS3PWN3ibsFVEJ8, Piece ID: 5RB4OO2XHF7WVT7HVTEQL4P4HAJYP3WRHDQVFWERDRBNKUVXYKSA) context deadline exceeded; piecestore: (Node ID: 1hz2v7h419qCvgzkvckp6PYDeQRKRGyHsLtyMGeG1VMa58A8EF, Piece ID: 2YOEIDEDQHBSYGLLEYYHV3BL63VCF2X55JPY5TKSBJSPOKUFEHHQ) context deadline exceeded; piecestore: (Node ID: 1uT7ntK6KAVDSErVdc37UcSd7xzCd3Dont23WitCYrA6KN4h4r, Piece ID: IGM33ZIHTUUKUXREFCY22HT7GRRK6RFODU7SU2N5ZXHWVIRO57KA) context deadline exceeded; piecestore: (Node ID: 1Ea8H6e3FPUzq42DSetCZ9e9ZZWtWpqyDy3fdYBucQDcCci8NN, Piece ID: WAZBXZ323RJVVK3IOKEHLLQJFA5LTNPQV72IP3GS3337GH4VGDAQ) context deadline exceeded; piecestore: (Node ID: 12ifo86KtZXZxEbUAxnvmX5tAg1C27mYub4VUTc9CkWTmpzPP3J, Piece ID: E4XT2PLZFP2HNL62PO5PMMXGQLMBDWV6FP63YSKKOV6LILK4EZLQ) context deadline exceeded; piecestore: (Node ID: 1APgLgqBaSndUajVXyuLWvkM2jbxwuYDkJhxVhweWNDjuju5A3, Piece ID: CDKE76XLSEISNM37HMKFG7STBORFP6NN47F326YW7B6HBEBGSZ4A) context deadline exceeded; piecestore: (Node ID: 12k5wzLZYUYDuQcbMv8NiXV2XBuZvbXbio3GErJYkzNEezwMwLG, Piece ID: TY4UXQXVTIAU2EWYBAFFPNM5OAQHJ5BJL24MGT7V5MSCGV4IDQ2Q) context deadline exceeded; piecestore: (Node ID: 12Ae31qNHzgX2cfrD8EVZqvJ9J95hijL4TEYtuGQPSpWNABG4kw, Piece ID: BIQOY7CGH7SDRD3YZHEBFNBIXUTAUXJMKHUOPTZIFSFXQCIVKSRA) context deadline exceeded; piecestore: (Node ID: 12EaBCW6FX8WY7HGhJJvDZrUy4QuBn8mH7xLz3v4BQA4rdKfKEg, Piece ID: ZJKUDMMW5TXZXRM63BHJ2IAOTQVRXQRMCLQFE6NYRQENBCI7VWIA) context deadline exceeded; piecestore: (Node ID: 12QzLS4YZYRfg6Aw82iFPkAioAuMn22Sf2EkwJFiZZsBDdvv1vS, Piece ID: HG7A6G3H3WOHBUO2QXHZZSYCYRPGZ3NW5MI7VE7F64GJHOHUQ34Q) context deadline exceeded; piecestore: (Node ID: 1vSFRya2YG5Agcz3txzeFfEvtymD4qGEBqrZjFYwEBR3L6tSdU, Piece ID: 5CBLLQNEOPVUGWZ3GYIXS65SDMJESBXYPPUUXH55HVRRKDGDYCPA) context deadline exceeded; piecestore: (Node ID: 12Tw2fxHJaQWpMnX3zZRtmp4Rpy9DEGx6vdZdo6x4eC6nvNgj7K, Piece ID: ULAMA6H4CRAUSN6KYQ2XXVJ4BMCJWB7WWTKBPTSQ7HF2IAZOHWNQ) context deadline exceeded; piecestore: (Node ID: 12aNknfsj5LsTiS8qb42nUSX1dYMhKbwesR6sZJiZtRJWzvfyqk, Piece ID: CKQCDBMNHUROXS34YWSXMJI4Z2A7DQHGKFTJBSGZ463YNGO62ZPA) context deadline exceeded; piecestore: (Node ID: 12UyANaqcy2NBjkmrA1buuHKpJcCxVz8vMpBzqujgmTQB78DY4u, Piece ID: POQJTXSQ45VBTFONHUXVLWMIY62MSQL6VZM6OOAHO42YIIZ6JJCA) context deadline exceeded; piecestore: (Node ID: 12YiXJaHTWUBq5fbKoFHHzeVxdSqpzeMb838cZgynEur4xHAEP, Piece ID: 2NRFXZUZJMBOCUXG6ZF2NRA4ILQ34UUPIPXHZE3QXRMAIWPUQTQA) context deadline exceeded; piecestore: (Node ID: 12uLGjR4eCbBC8GY5vPc4UJmj4AkzGxQd7WQeSo5aw4AxbpaF6J, Piece ID: XGCOLCZIHEWZAJHYAX45DH5LISJCNAWTPHRNM6Q2IZ3UC4TRDDZA) context deadline exceeded; piecestore: (Node ID: 12ntDfA4LBBPkXUioZRWwQhMMA7kBN5rpMd91BgMi7tw2uuRi7n, Piece ID: 4D7CGCOOSMD6WHPV6PRLKDAME6A6SKX3NT32XS2KLYZKO2CJTURA) context deadline exceeded; piecestore: (Node ID: 12QjxocGH8nPtfBmZqTVmV3sssQs5nR8QsWaVga3BdKU97XgPBz, Piece ID: O75ISG3WWM46CC4ZALCC56AAVRXMRPYUMUOMWZVIEWNTLQ5B6UVQ) context deadline exceeded; piecestore: (Node ID: 1qEoNfbFCs5TobVBDG2aV1qSNU6At8ZUxSeGGaYFytgCcArXg7, Piece ID: VNDX43FYBGVJMPQ4773RC5OSYOPTJTYD26RI37ULIRNZKKLATSZA) context deadline exceeded; piecestore: (Node ID: 12hRf4FXCVVxzPjwRGscsH6Ei7Cgo3EQciSDH99PCNonSP9vTVE, Piece ID: YG22O5IGTGUBMAAODU3T7EBKRLRATKLYBNVSRAZ7APWNIAL2NXKA) context deadline exceeded; piecestore: (Node ID: 15KQ2SjAnXeXBccMHNKZff2PLjeLd2a1WNFtk1ebx1GBdhd3kw, Piece ID: VZWPUBPHQRLPE67H7PPWBO4LGQ7ANOZAFJY3P6VIGQEYJZI5DYYA) context deadline exceeded; piecestore: (Node ID: 12eEQzjo5uQZGL1JR2AyinqCXxderwuqTKvTmoFZ3uNDqwE6WAo, Piece ID: E2SFJCYFLH6UQZ5BZN2YOB3NKEMLRGAJB2GPIXIUBBK53P7VNDOQ) context deadline exceeded; piecestore: (Node ID: 12vQvbGahDxH7tm6NWygYvfuCerQaLqA3BQNEG7bSDKgEKRbUaJ, Piece ID: YTRN7GBUWG5QQIYBU4RB7VK7LS2PTRXTV44BN7P5GCO2VF5F5D2A) context deadline exceeded; piecestore: (Node ID: 12XCy3WvXLNmgSisHzPVy8MFWS8jiA5gnTNTiux7s4FaLnaJWAU, Piece ID: ZD4QR2TJPVJLE545HQGYDR5NDZPEWUMQISU3A2PNTVBE7WH6AU6A) context deadline exceeded Duration: 4.958678142s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.005841001s trace: https://jaeger.dev.storj.io/trace/696a333689b50b84 FAIL (2025-12-07 22:40:55 UTC) orbiter warning: operation: sync2.Copy uplink: piecestore: (Node ID: 1ByMj85eQ9yd81bpEjSKrqFTEnLsuV132WoAMBxHrd12jgqddx, Piece ID: PK5SS7JPDNWVGZ4EJ43YWL4MNKGSCS7564EM62UJPBRHXKB4KBHA) context deadline exceeded; piecestore: (Node ID: 1LmXgEBf4pEfYehAES7kR3sVLRt7QxS2pvttiNtqaDnYp7PDzy, Piece ID: Z7PBN4D4PZIWZZZDO7A666MCKEUIZ7ZG3O64BVY3YITRZ7SJISEQ) context deadline exceeded; piecestore: (Node ID: 12Hq4E5wWAPzrxZacP6gbFpKiBvbRXnPdSR4z9jRU443yCFqS2r, Piece ID: CAAZO3SF5Y2YQL2VXBEONDARGL3AXYRZWJSFH3CMANA4J5GOHD7Q) context deadline exceeded; piecestore: (Node ID: 12SQdxBgr7pu6Kz3M16tJjezo5bMiG1gKB57kS3PWN3ibsFVEJ8, Piece ID: 5RB4OO2XHF7WVT7HVTEQL4P4HAJYP3WRHDQVFWERDRBNKUVXYKSA) context deadline exceeded; piecestore: (Node ID: 1hz2v7h419qCvgzkvckp6PYDeQRKRGyHsLtyMGeG1VMa58A8EF, Piece ID: 2YOEIDEDQHBSYGLLEYYHV3BL63VCF2X55JPY5TKSBJSPOKUFEHHQ) context deadline exceeded; piecestore: (Node ID: 1uT7ntK6KAVDSErVdc37UcSd7xzCd3Dont23WitCYrA6KN4h4r, Piece ID: IGM33ZIHTUUKUXREFCY22HT7GRRK6RFODU7SU2N5ZXHWVIRO57KA) context deadline exceeded; piecestore: (Node ID: 1Ea8H6e3FPUzq42DSetCZ9e9ZZWtWpqyDy3fdYBucQDcCci8NN, Piece ID: WAZBXZ323RJVVK3IOKEHLLQJFA5LTNPQV72IP3GS3337GH4VGDAQ) context deadline exceeded; piecestore: (Node ID: 12ifo86KtZXZxEbUAxnvmX5tAg1C27mYub4VUTc9CkWTmpzPP3J, Piece ID: E4XT2PLZFP2HNL62PO5PMMXGQLMBDWV6FP63YSKKOV6LILK4EZLQ) context deadline exceeded; piecestore: (Node ID: 1APgLgqBaSndUajVXyuLWvkM2jbxwuYDkJhxVhweWNDjuju5A3, Piece ID: CDKE76XLSEISNM37HMKFG7STBORFP6NN47F326YW7B6HBEBGSZ4A) context deadline exceeded; piecestore: (Node ID: 12k5wzLZYUYDuQcbMv8NiXV2XBuZvbXbio3GErJYkzNEezwMwLG, Piece ID: TY4UXQXVTIAU2EWYBAFFPNM5OAQHJ5BJL24MGT7V5MSCGV4IDQ2Q) context deadline exceeded; piecestore: (Node ID: 12Ae31qNHzgX2cfrD8EVZqvJ9J95hijL4TEYtuGQPSpWNABG4kw, Piece ID: BIQOY7CGH7SDRD3YZHEBFNBIXUTAUXJMKHUOPTZIFSFXQCIVKSRA) context deadline exceeded; piecestore: (Node ID: 12EaBCW6FX8WY7HGhJJvDZrUy4QuBn8mH7xLz3v4BQA4rdKfKEg, Piece ID: ZJKUDMMW5TXZXRM63BHJ2IAOTQVRXQRMCLQFE6NYRQENBCI7VWIA) context deadline exceeded; piecestore: (Node ID: 12QzLS4YZYRfg6Aw82iFPkAioAuMn22Sf2EkwJFiZZsBDdvv1vS, Piece ID: HG7A6G3H3WOHBUO2QXHZZSYCYRPGZ3NW5MI7VE7F64GJHOHUQ34Q) context deadline exceeded; piecestore: (Node ID: 1vSFRya2YG5Agcz3txzeFfEvtymD4qGEBqrZjFYwEBR3L6tSdU, Piece ID: 5CBLLQNEOPVUGWZ3GYIXS65SDMJESBXYPPUUXH55HVRRKDGDYCPA) context deadline exceeded; piecestore: (Node ID: 12Tw2fxHJaQWpMnX3zZRtmp4Rpy9DEGx6vdZdo6x4eC6nvNgj7K, Piece ID: ULAMA6H4CRAUSN6KYQ2XXVJ4BMCJWB7WWTKBPTSQ7HF2IAZOHWNQ) context deadline exceeded; piecestore: (Node ID: 12aNknfsj5LsTiS8qb42nUSX1dYMhKbwesR6sZJiZtRJWzvfyqk, Piece ID: CKQCDBMNHUROXS34YWSXMJI4Z2A7DQHGKFTJBSGZ463YNGO62ZPA) context deadline exceeded; piecestore: (Node ID: 12UyANaqcy2NBjkmrA1buuHKpJcCxVz8vMpBzqujgmTQB78DY4u, Piece ID: POQJTXSQ45VBTFONHUXVLWMIY62MSQL6VZM6OOAHO42YIIZ6JJCA) context deadline exceeded; piecestore: (Node ID: 12YiXJaHTWUBq5fbKoFHHzeVxdSqpzeMb838cZgynEur4xHAEP, Piece ID: 2NRFXZUZJMBOCUXG6ZF2NRA4ILQ34UUPIPXHZE3QXRMAIWPUQTQA) context deadline exceeded; piecestore: (Node ID: 12uLGjR4eCbBC8GY5vPc4UJmj4AkzGxQd7WQeSo5aw4AxbpaF6J, Piece ID: XGCOLCZIHEWZAJHYAX45DH5LISJCNAWTPHRNM6Q2IZ3UC4TRDDZA) context deadline exceeded; piecestore: (Node ID: 12ntDfA4LBBPkXUioZRWwQhMMA7kBN5rpMd91BgMi7tw2uuRi7n, Piece ID: 4D7CGCOOSMD6WHPV6PRLKDAME6A6SKX3NT32XS2KLYZKO2CJTURA) context deadline exceeded; piecestore: (Node ID: 12QjxocGH8nPtfBmZqTVmV3sssQs5nR8QsWaVga3BdKU97XgPBz, Piece ID: O75ISG3WWM46CC4ZALCC56AAVRXMRPYUMUOMWZVIEWNTLQ5B6UVQ) context deadline exceeded; piecestore: (Node ID: 1qEoNfbFCs5TobVBDG2aV1qSNU6At8ZUxSeGGaYFytgCcArXg7, Piece ID: VNDX43FYBGVJMPQ4773RC5OSYOPTJTYD26RI37ULIRNZKKLATSZA) context deadline exceeded; piecestore: (Node ID: 12hRf4FXCVVxzPjwRGscsH6Ei7Cgo3EQciSDH99PCNonSP9vTVE, Piece ID: YG22O5IGTGUBMAAODU3T7EBKRLRATKLYBNVSRAZ7APWNIAL2NXKA) context deadline exceeded; piecestore: (Node ID: 15KQ2SjAnXeXBccMHNKZff2PLjeLd2a1WNFtk1ebx1GBdhd3kw, Piece ID: VZWPUBPHQRLPE67H7PPWBO4LGQ7ANOZAFJY3P6VIGQEYJZI5DYYA) context deadline exceeded; piecestore: (Node ID: 12eEQzjo5uQZGL1JR2AyinqCXxderwuqTKvTmoFZ3uNDqwE6WAo, Piece ID: E2SFJCYFLH6UQZ5BZN2YOB3NKEMLRGAJB2GPIXIUBBK53P7VNDOQ) context deadline exceeded; piecestore: (Node ID: 12vQvbGahDxH7tm6NWygYvfuCerQaLqA3BQNEG7bSDKgEKRbUaJ, Piece ID: YTRN7GBUWG5QQIYBU4RB7VK7LS2PTRXTV44BN7P5GCO2VF5F5D2A) context deadline exceeded; piecestore: (Node ID: 12XCy3WvXLNmgSisHzPVy8MFWS8jiA5gnTNTiux7s4FaLnaJWAU, Piece ID: ZD4QR2TJPVJLE545HQGYDR5NDZPEWUMQISU3A2PNTVBE7WH6AU6A) context deadline exceeded Duration: 30.005870648s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.740669713s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:48:48 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.740694038s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.693146917s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:48:00 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.693174932s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.797726428s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:47:02 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.797770583s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.09212428s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:46:05 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 6.092157433s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.747262865s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:45:17 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.747289847s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.710628036s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:44:30 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.710660047s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.646390749s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:43:34 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.646419383s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.717333095s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:42:26 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.717359006s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.608536349s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:41:15 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.608562448s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.677916401s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:40:01 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.67795303s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.81358175s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:39:01 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.813611979s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 5.637300525s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:38:38 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.637333086s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.756714237s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:37:40 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.756763601s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.90163346s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:37:07 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.901661914s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 5.756969378s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:36:35 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.757000337s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.696116792s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:36:10 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.696143783s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 5.874913554s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:35:10 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.874935435s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.719648239s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:34:40 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.719680832s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.724705361s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:34:12 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.724735298s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.757220388s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:33:50 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.757249462s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.66608104s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:33:08 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.666110948s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.757972833s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:31:58 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.75807678s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.90754806s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:30:54 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.907585852s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.712193329s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:30:29 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.712222715s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.792345983s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:30:02 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.792369037s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.743875305s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:29:23 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.743905351s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 5.793609947s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:28:30 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.793643761s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.920257256s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:27:30 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.920303945s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.687102122s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:26:43 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.687141728s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.742867694s trace: https://jaeger.dev.storj.io/trace/2dbb9d950879d488 FAIL (2025-12-07 17:25:36 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 7 17:04:40.750068351 (49h35m40.621151396s) [upload=43608]: Starting upload Dec 7 17:04:40.750151529 (49h35m40.621234574s) [upload=43608]: Done starting upload Dec 7 17:04:40.750252862 (49h35m40.621335927s) [upload=43608]: Uploading segments... Dec 7 17:04:40.750449186 (49h35m40.621532241s) [upload=43608]: Got next segment. Inline: false Dec 7 17:04:40.750495423 (49h35m40.621578459s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 7 17:04:40.750502757 (49h35m40.621585802s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 7 17:04:41.012862673 (49h35m40.883945718s) [seg_pos={0 0}, upload=43608]: Begin upload segment... Dec 7 17:04:41.012921534 (49h35m40.88400458s) [seg_pos={0 0}, upload=43608]: Done begin upload segment. Dec 7 17:04:41.012952173 (49h35m40.884035218s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.014639106 (49h35m40.885722151s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.015553246 (49h35m40.886636291s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016060653 (49h35m40.887143698s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.016721020 (49h35m40.887804065s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.018348218 (49h35m40.889431273s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.020177682 (49h35m40.891260727s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.021303415 (49h35m40.89238646s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.023085799 (49h35m40.894168844s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.024447463 (49h35m40.895530498s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.025964030 (49h35m40.897047065s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.027289063 (49h35m40.898372108s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.028891215 (49h35m40.89997425s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.031185423 (49h35m40.902268468s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.032762747 (49h35m40.903845782s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:41.034246312 (49h35m40.905329347s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Uploading piece... Dec 7 17:04:42.710512985 (49h35m42.58159602s) [upload=43608]: Next returned nil segment Dec 7 17:04:42.710525298 (49h35m42.581608343s) [upload=43608]: Waiting for error group managing segments... Dec 7 17:04:43.179511201 (49h35m43.050594246s) [noise=true , address=77.90.31.15:4699 , piece_id=LDYETSF6H3QRLQP3R5K5ND23WNM6OSCLKT2UNVYDFUWLAXBUAYAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:46.586841777 (49h35m46.457924822s) [noise=true , address=145.79.227.43:4158 , piece_id=UATURKONJQYIHHK5V7JBAM22Q6CAFZVZY2EGLIES2XK4M5QXVFBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:53.671123584 (49h35m53.542206629s) [noise=true , address=145.79.227.28:4725 , piece_id=DEZB2ZETZPFK4JQWARH2BMOBWTHB3LJNGUEI3HCMUIPHBCVRISTQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:55.169666550 (49h35m55.040749595s) [noise=true , address=145.79.227.28:4727 , piece_id=MXQMYHTX43GIYL7QXMJGRKP3CUWVI77UIN55TDNXQ42Z2RR4XCMQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:04:57.066377404 (49h35m56.937460449s) [noise=true , address=145.79.227.40:4050 , piece_id=4LP7RUN433VLXH25GVIBF4KSTJ2AB4JC66BZUTHQRXBMZXBGFGDQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:00.087003610 (49h35m59.958086645s) [noise=true , address=145.79.227.34:4907 , piece_id=GCHR4OE2RJTFC4CXKB2KAPCFRUCCX6EHOTCLKLGBYZROFFDYYZ4Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.688144215 (49h36m2.55922726s) [noise=true , address=145.79.227.29:4941 , piece_id=VOAWSTMHYNYRCHGCSEDLS26KZPXN6DQLUFHJF75VNMIY2JHRS6VA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:02.873597685 (49h36m2.74468073s) [noise=true , address=145.79.227.38:4979 , piece_id=GJBNNFOYPO5Y6ERMDEASIF2KT3X3CVESACNV3X55GM2UEGG4NPEA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:03.747085322 (49h36m3.618168377s) [noise=true , address=145.79.227.44:4195 , piece_id=NXTYROFR2TDL7G6PYNXNIWW66C4OMDSPRDZMYSO5Q22DOPAAMNUA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.570204756 (49h36m4.441287801s) [noise=true , address=145.79.227.23:4546 , piece_id=L5GZIGQNRV42D5JUSSEHZGLZGRWVQHCRJZSXSUNABM2ZK5LNHVBA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:04.766016562 (49h36m4.637099607s) [noise=true , address=145.79.227.33:4869 , piece_id=BZXWNMKNHTJGSHX5RAB2IWPM2AAP6ZXFLQ4726LZDRGAHPZAXD2Q, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:06.426653835 (49h36m6.29773689s) [noise=true , address=145.79.227.39:4013 , piece_id=SHEFK4QV4AOXHBEUKQ5LFDIDBJRRMAAW7I3BFGOIOFVI4UA3LB4A, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621680284 (49h36m7.492763339s) [noise=true , address=145.79.227.42:4121 , piece_id=4T7DFOI2TSKKZDADMZAY57IU2K3AC63O3MPXN26OK2ODK6NYHWOA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: <nil> Dec 7 17:05:07.621710732 (49h36m7.492793777s) [seg_pos={0 0}, upload=43608]: Segment reached optimal threshold of 13 pieces. Dec 7 17:05:07.622110042 (49h36m7.493193097s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fyCro9gG2d3myk6Cju5vmf4CbkqbJN2PKAP8EbpcT1Rtao27H): protocol: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53898->145.79.227.27:4690: use of closed network connection Dec 7 17:05:07.622121895 (49h36m7.49320494s) [noise=true , address=145.79.227.27:4690 , piece_id=Q7KXIDF46P6ZIYRZ2URIFK6ZD3JBKIPWRSMXJAPIBPJT4VGHBJNA, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622169114 (49h36m7.49325216s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz): protocol: context canceled; context canceled Dec 7 17:05:07.622181257 (49h36m7.493264302s) [noise=true , address=145.79.227.43:4157 , piece_id=2BWIWME4ZI3IIDQKCUWCMYE3E3NNW3X4WUYBG6KAHB63ZQAEAZAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622414712 (49h36m7.493497757s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: read tcp 188.166.225.170:41952->145.79.227.26:4653: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 7 17:05:07.622423679 (49h36m7.493506724s) [noise=true , address=145.79.227.26:4653 , piece_id=65YZ4VCBYBFYFW7AVODMDRMYMRF4DEMQMMJVJQGSRZEML5QQ7KAQ, seg_pos={0 0}, upload=43608]: Long tail cancellation due to optimal threshold Dec 7 17:05:07.622449889 (49h36m7.493532924s) [seg_pos={0 0}, upload=43608]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 7 17:05:07.622527457 (49h36m7.493610502s) [upload=43608]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 7 17:05:07.622532175 (49h36m7.49361522s) [upload=43608]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 7 17:05:07.933366757 (49h36m7.804449802s) [upload=43608]: Done uploading segments... Dec 7 17:05:07.933395712 (49h36m7.804478757s) [upload=43608]: Upload finished. err: <nil> Duration: 5.74289753s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)