Orbiter History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 1.439555602s trace: https://jaeger.dev.storj.io/trace/6ea3fd74aff47917 FAIL (2025-12-22 16:26:27 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.43958087s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X............................X.........................] duration: 5.597954521s trace: https://jaeger.dev.storj.io/trace/72726d8466c916ea FAIL (2025-12-22 16:26:26 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 5.597982025s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [..X..XXXX....X..X...XX.....XXXX......X.X.X.X...X....XXXXXXX.] duration: 10.52597818s trace: https://jaeger.dev.storj.io/trace/7fc09410a4e58 FAIL (2025-12-22 16:26:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, Piece ID: YDPRLTLHQFZFNN2MDDXJ47RP5BBVXKIZSXFJSIXUXDKIYTNVILCQ) context deadline exceeded; piecestore: (Node ID: 128f6i7ALVXF3Vr8G8zsZXrNBFiT3cK8UAiic5XbFS1gRjZK86m, Piece ID: IBQEIAODVZGVHWPYB6SZ2WLJC5GBU2VFDMGRFZI2WFGYNKQQJ2HQ) context deadline exceeded ======LOGS====== Dec 22 16:25:40.749205293 (408h56m40.620288328s) [upload=359010]: Starting upload Dec 22 16:25:40.749262994 (408h56m40.620346029s) [upload=359010]: Done starting upload Dec 22 16:25:40.749458596 (408h56m40.620541641s) [upload=359010]: Uploading segments... Dec 22 16:25:40.749551773 (408h56m40.620634808s) [upload=359010]: Got next segment. Inline: false Dec 22 16:25:40.749587561 (408h56m40.620670596s) [upload=359010]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 16:25:40.749593744 (408h56m40.620676779s) [upload=359010]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 16:25:41.080419648 (408h56m40.951502683s) [seg_pos={0 0}, upload=359010]: Begin upload segment... Dec 22 16:25:41.080467358 (408h56m40.951550393s) [seg_pos={0 0}, upload=359010]: Done begin upload segment. Dec 22 16:25:41.080540147 (408h56m40.951623182s) [noise=true , address=145.79.227.46:4603 , piece_id=XXP4HUASFRX6TVFBTXMUHEU5I77YAXOEMCBI3N5NH2TSMAIOMN5A, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.080554926 (408h56m40.951637971s) [noise=true , address=145.79.227.11:4732 , piece_id=D4IPGGRMQB2HZJUNVCSCV5KCD2VH2FE72B4FUNNM56JU2WMQ3CXA, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.081117777 (408h56m40.952200812s) [noise=true , address=145.79.227.43:4157 , piece_id=YDS6YME6V37C55NLNH4P3YW6GMF5UFBJTP33GVTLWQ5MBXTEZTFQ, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.081453456 (408h56m40.952536491s) [noise=true , address=145.79.227.35:4481 , piece_id=P2RXVG2UOSBVDPI3RM7JQI3AVOSGVGG7SJ3B2AM2WCJZATT2YAWA, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.081730183 (408h56m40.952813218s) [noise=true , address=145.79.227.19:4078 , piece_id=DPQ4PUJK7AJPDT5HNTKIL3RXFF3357TT7BT7B4YQKBAX52T2MRBA, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.081910687 (408h56m40.952993732s) [noise=true , address=145.79.227.49:4338 , piece_id=KAEYGZFDIW2L4YRUOW5XCCIWFTISQA4ORGE3UHO3G55YTB4P54UQ, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.082008793 (408h56m40.953091828s) [noise=true , address=145.79.227.31:4799 , piece_id=KMPAWOZTTYLUAVFQ4FO5XL6J2WV6LTYMKQ4NN2JOQHFJ7PHJDNMQ, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.082198886 (408h56m40.953281921s) [noise=true , address=45.91.116.11:4118 , piece_id=J2G4RBUUKNZ2H7TYJ234OTM2PECY66TRFEBRTJ6NQC2RT2BOEKFQ, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.082285991 (408h56m40.953369036s) [noise=true , address=145.79.227.45:4229 , piece_id=LHMUNX6RFSMINZBT273YCG5DXXJDKZL6O4PHBP3CUZQGVSXI3ZKA, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.082548872 (408h56m40.953631907s) [noise=true , address=145.79.227.51:4411 , piece_id=ICNWN2QTSMN7HEBL3NKB3US4Q6AH6RZZVLMEC6R7VSVV4P3UYHJQ, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.082822333 (408h56m40.953905378s) [noise=true , address=145.79.227.37:4565 , piece_id=CGUBWJTUNAHCPSFDOJ4SK4ESMDQBLXCBSXG64GL4GTG35NS2FNDQ, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.083132824 (408h56m40.954215859s) [noise=true , address=45.91.116.12:4225 , piece_id=2DA5MBVLDWJHJHQUB6V7HMEMMIANHLPIKCGMXEO22DZ632MRGRXQ, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.083409080 (408h56m40.954492116s) [noise=true , address=77.90.31.19:4842 , piece_id=NMEOAL3GYOAAQFFTN7V7MFBD5D6DBZ2A2ZKAFYDNSX5U6TWDIBWA, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.083806156 (408h56m40.954889191s) [noise=true , address=145.79.227.10:4401 , piece_id=RNLQMUK2ATVQXAKHVRF4KHX4ENQWCOYICYXYLPAGZUK2NXTWCW5Q, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.084163657 (408h56m40.955246702s) [noise=true , address=77.90.31.17:4773 , piece_id=YDPRLTLHQFZFNN2MDDXJ47RP5BBVXKIZSXFJSIXUXDKIYTNVILCQ, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.084482163 (408h56m40.955565208s) [noise=true , address=45.91.116.9:4264 , piece_id=IBQEIAODVZGVHWPYB6SZ2WLJC5GBU2VFDMGRFZI2WFGYNKQQJ2HQ, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:42.864405931 (408h56m42.735488966s) [upload=359010]: Next returned nil segment Dec 22 16:25:42.864419486 (408h56m42.735502521s) [upload=359010]: Waiting for error group managing segments... Dec 22 16:25:43.283368762 (408h56m43.154451797s) [noise=true , address=77.90.31.17:4773 , piece_id=YDPRLTLHQFZFNN2MDDXJ47RP5BBVXKIZSXFJSIXUXDKIYTNVILCQ, seg_pos={0 0}, upload=359010]: Done uploading piece. err: <nil> Dec 22 16:25:43.345461266 (408h56m43.216544301s) [noise=true , address=77.90.31.19:4842 , piece_id=NMEOAL3GYOAAQFFTN7V7MFBD5D6DBZ2A2ZKAFYDNSX5U6TWDIBWA, seg_pos={0 0}, upload=359010]: Done uploading piece. err: <nil> Dec 22 16:25:43.430530702 (408h56m43.301613737s) [noise=true , address=45.91.116.12:4225 , piece_id=2DA5MBVLDWJHJHQUB6V7HMEMMIANHLPIKCGMXEO22DZ632MRGRXQ, seg_pos={0 0}, upload=359010]: Done uploading piece. err: <nil> Dec 22 16:25:43.607233011 (408h56m43.478316046s) [noise=true , address=145.79.227.45:4229 , piece_id=LHMUNX6RFSMINZBT273YCG5DXXJDKZL6O4PHBP3CUZQGVSXI3ZKA, seg_pos={0 0}, upload=359010]: Done uploading piece. err: <nil> Dec 22 16:25:43.615452442 (408h56m43.486535487s) [noise=true , address=145.79.227.35:4481 , piece_id=P2RXVG2UOSBVDPI3RM7JQI3AVOSGVGG7SJ3B2AM2WCJZATT2YAWA, seg_pos={0 0}, upload=359010]: Done uploading piece. err: <nil> Dec 22 16:25:43.616375138 (408h56m43.487458173s) [noise=true , address=145.79.227.31:4799 , piece_id=KMPAWOZTTYLUAVFQ4FO5XL6J2WV6LTYMKQ4NN2JOQHFJ7PHJDNMQ, seg_pos={0 0}, upload=359010]: Done uploading piece. err: <nil> Dec 22 16:25:43.625864228 (408h56m43.496947273s) [noise=true , address=145.79.227.46:4603 , piece_id=XXP4HUASFRX6TVFBTXMUHEU5I77YAXOEMCBI3N5NH2TSMAIOMN5A, seg_pos={0 0}, upload=359010]: Done uploading piece. err: <nil> Dec 22 16:25:43.633964582 (408h56m43.505047617s) [noise=true , address=145.79.227.37:4565 , piece_id=CGUBWJTUNAHCPSFDOJ4SK4ESMDQBLXCBSXG64GL4GTG35NS2FNDQ, seg_pos={0 0}, upload=359010]: Done uploading piece. err: <nil> Dec 22 16:25:43.634530840 (408h56m43.505613875s) [noise=true , address=145.79.227.43:4157 , piece_id=YDS6YME6V37C55NLNH4P3YW6GMF5UFBJTP33GVTLWQ5MBXTEZTFQ, seg_pos={0 0}, upload=359010]: Done uploading piece. err: <nil> Dec 22 16:25:43.656709091 (408h56m43.527792126s) [noise=true , address=145.79.227.51:4411 , piece_id=ICNWN2QTSMN7HEBL3NKB3US4Q6AH6RZZVLMEC6R7VSVV4P3UYHJQ, seg_pos={0 0}, upload=359010]: Done uploading piece. err: <nil> Dec 22 16:25:43.677506783 (408h56m43.548589818s) [noise=true , address=45.91.116.9:4264 , piece_id=IBQEIAODVZGVHWPYB6SZ2WLJC5GBU2VFDMGRFZI2WFGYNKQQJ2HQ, seg_pos={0 0}, upload=359010]: Done uploading piece. err: <nil> Dec 22 16:25:43.867878541 (408h56m43.738961586s) [noise=true , address=145.79.227.19:4078 , piece_id=DPQ4PUJK7AJPDT5HNTKIL3RXFF3357TT7BT7B4YQKBAX52T2MRBA, seg_pos={0 0}, upload=359010]: Done uploading piece. err: <nil> Dec 22 16:25:43.868572843 (408h56m43.739655878s) [noise=true , address=145.79.227.11:4732 , piece_id=D4IPGGRMQB2HZJUNVCSCV5KCD2VH2FE72B4FUNNM56JU2WMQ3CXA, seg_pos={0 0}, upload=359010]: Done uploading piece. err: <nil> Dec 22 16:25:43.868584325 (408h56m43.73966737s) [seg_pos={0 0}, upload=359010]: Segment reached optimal threshold of 13 pieces. Dec 22 16:25:43.868820244 (408h56m43.739903279s) [noise=true , address=145.79.227.10:4401 , piece_id=RNLQMUK2ATVQXAKHVRF4KHX4ENQWCOYICYXYLPAGZUK2NXTWCW5Q, seg_pos={0 0}, upload=359010]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1): protocol: expected piece hash; context canceled; EOF Dec 22 16:25:43.868829802 (408h56m43.739912847s) [noise=true , address=145.79.227.10:4401 , piece_id=RNLQMUK2ATVQXAKHVRF4KHX4ENQWCOYICYXYLPAGZUK2NXTWCW5Q, seg_pos={0 0}, upload=359010]: Long tail cancellation due to optimal threshold Dec 22 16:25:43.869117540 (408h56m43.740200585s) [noise=true , address=45.91.116.11:4118 , piece_id=J2G4RBUUKNZ2H7TYJ234OTM2PECY66TRFEBRTJ6NQC2RT2BOEKFQ, seg_pos={0 0}, upload=359010]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Dec 22 16:25:43.869126488 (408h56m43.740209533s) [noise=true , address=45.91.116.11:4118 , piece_id=J2G4RBUUKNZ2H7TYJ234OTM2PECY66TRFEBRTJ6NQC2RT2BOEKFQ, seg_pos={0 0}, upload=359010]: Long tail cancellation due to optimal threshold Dec 22 16:25:43.869300800 (408h56m43.740383835s) [noise=true , address=145.79.227.49:4338 , piece_id=KAEYGZFDIW2L4YRUOW5XCCIWFTISQA4ORGE3UHO3G55YTB4P54UQ, seg_pos={0 0}, upload=359010]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN): protocol: expected piece hash; context canceled; EOF Dec 22 16:25:43.869308975 (408h56m43.74039202s) [noise=true , address=145.79.227.49:4338 , piece_id=KAEYGZFDIW2L4YRUOW5XCCIWFTISQA4ORGE3UHO3G55YTB4P54UQ, seg_pos={0 0}, upload=359010]: Long tail cancellation due to optimal threshold Dec 22 16:25:43.869327901 (408h56m43.740410957s) [seg_pos={0 0}, upload=359010]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 22 16:25:43.869403946 (408h56m43.740486991s) [upload=359010]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 16:25:43.869410508 (408h56m43.740493553s) [upload=359010]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 16:25:44.290535067 (408h56m44.161618172s) [upload=359010]: Done uploading segments... Dec 22 16:25:44.290561687 (408h56m44.161644762s) [upload=359010]: Upload finished. err: <nil> Duration: 10.525998308s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X............................X........................] duration: 5.878034099s trace: https://jaeger.dev.storj.io/trace/72726d8466c916ea FAIL (2025-12-22 16:26:11 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 5.878062602s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [...X..XXXX....X..X...XX.....XXXX......X.X.X.X...X....XXXXXXX] duration: 30.004237732s trace: https://jaeger.dev.storj.io/trace/7fc09410a4e58 FAIL (2025-12-22 16:26:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, Piece ID: YDPRLTLHQFZFNN2MDDXJ47RP5BBVXKIZSXFJSIXUXDKIYTNVILCQ) context deadline exceeded; piecestore: (Node ID: 128f6i7ALVXF3Vr8G8zsZXrNBFiT3cK8UAiic5XbFS1gRjZK86m, Piece ID: IBQEIAODVZGVHWPYB6SZ2WLJC5GBU2VFDMGRFZI2WFGYNKQQJ2HQ) context deadline exceeded ======LOGS====== Dec 22 16:25:40.749205293 (408h56m40.620288328s) [upload=359010]: Starting upload Dec 22 16:25:40.749262994 (408h56m40.620346029s) [upload=359010]: Done starting upload Dec 22 16:25:40.749458596 (408h56m40.620541641s) [upload=359010]: Uploading segments... Dec 22 16:25:40.749551773 (408h56m40.620634808s) [upload=359010]: Got next segment. Inline: false Dec 22 16:25:40.749587561 (408h56m40.620670596s) [upload=359010]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 16:25:40.749593744 (408h56m40.620676779s) [upload=359010]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 16:25:41.080419648 (408h56m40.951502683s) [seg_pos={0 0}, upload=359010]: Begin upload segment... Dec 22 16:25:41.080467358 (408h56m40.951550393s) [seg_pos={0 0}, upload=359010]: Done begin upload segment. Dec 22 16:25:41.080540147 (408h56m40.951623182s) [noise=true , address=145.79.227.46:4603 , piece_id=XXP4HUASFRX6TVFBTXMUHEU5I77YAXOEMCBI3N5NH2TSMAIOMN5A, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.080554926 (408h56m40.951637971s) [noise=true , address=145.79.227.11:4732 , piece_id=D4IPGGRMQB2HZJUNVCSCV5KCD2VH2FE72B4FUNNM56JU2WMQ3CXA, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.081117777 (408h56m40.952200812s) [noise=true , address=145.79.227.43:4157 , piece_id=YDS6YME6V37C55NLNH4P3YW6GMF5UFBJTP33GVTLWQ5MBXTEZTFQ, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.081453456 (408h56m40.952536491s) [noise=true , address=145.79.227.35:4481 , piece_id=P2RXVG2UOSBVDPI3RM7JQI3AVOSGVGG7SJ3B2AM2WCJZATT2YAWA, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.081730183 (408h56m40.952813218s) [noise=true , address=145.79.227.19:4078 , piece_id=DPQ4PUJK7AJPDT5HNTKIL3RXFF3357TT7BT7B4YQKBAX52T2MRBA, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.081910687 (408h56m40.952993732s) [noise=true , address=145.79.227.49:4338 , piece_id=KAEYGZFDIW2L4YRUOW5XCCIWFTISQA4ORGE3UHO3G55YTB4P54UQ, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.082008793 (408h56m40.953091828s) [noise=true , address=145.79.227.31:4799 , piece_id=KMPAWOZTTYLUAVFQ4FO5XL6J2WV6LTYMKQ4NN2JOQHFJ7PHJDNMQ, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.082198886 (408h56m40.953281921s) [noise=true , address=45.91.116.11:4118 , piece_id=J2G4RBUUKNZ2H7TYJ234OTM2PECY66TRFEBRTJ6NQC2RT2BOEKFQ, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.082285991 (408h56m40.953369036s) [noise=true , address=145.79.227.45:4229 , piece_id=LHMUNX6RFSMINZBT273YCG5DXXJDKZL6O4PHBP3CUZQGVSXI3ZKA, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.082548872 (408h56m40.953631907s) [noise=true , address=145.79.227.51:4411 , piece_id=ICNWN2QTSMN7HEBL3NKB3US4Q6AH6RZZVLMEC6R7VSVV4P3UYHJQ, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.082822333 (408h56m40.953905378s) [noise=true , address=145.79.227.37:4565 , piece_id=CGUBWJTUNAHCPSFDOJ4SK4ESMDQBLXCBSXG64GL4GTG35NS2FNDQ, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.083132824 (408h56m40.954215859s) [noise=true , address=45.91.116.12:4225 , piece_id=2DA5MBVLDWJHJHQUB6V7HMEMMIANHLPIKCGMXEO22DZ632MRGRXQ, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.083409080 (408h56m40.954492116s) [noise=true , address=77.90.31.19:4842 , piece_id=NMEOAL3GYOAAQFFTN7V7MFBD5D6DBZ2A2ZKAFYDNSX5U6TWDIBWA, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.083806156 (408h56m40.954889191s) [noise=true , address=145.79.227.10:4401 , piece_id=RNLQMUK2ATVQXAKHVRF4KHX4ENQWCOYICYXYLPAGZUK2NXTWCW5Q, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.084163657 (408h56m40.955246702s) [noise=true , address=77.90.31.17:4773 , piece_id=YDPRLTLHQFZFNN2MDDXJ47RP5BBVXKIZSXFJSIXUXDKIYTNVILCQ, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:41.084482163 (408h56m40.955565208s) [noise=true , address=45.91.116.9:4264 , piece_id=IBQEIAODVZGVHWPYB6SZ2WLJC5GBU2VFDMGRFZI2WFGYNKQQJ2HQ, seg_pos={0 0}, upload=359010]: Uploading piece... Dec 22 16:25:42.864405931 (408h56m42.735488966s) [upload=359010]: Next returned nil segment Dec 22 16:25:42.864419486 (408h56m42.735502521s) [upload=359010]: Waiting for error group managing segments... Dec 22 16:25:43.283368762 (408h56m43.154451797s) [noise=true , address=77.90.31.17:4773 , piece_id=YDPRLTLHQFZFNN2MDDXJ47RP5BBVXKIZSXFJSIXUXDKIYTNVILCQ, seg_pos={0 0}, upload=359010]: Done uploading piece. err: <nil> Dec 22 16:25:43.345461266 (408h56m43.216544301s) [noise=true , address=77.90.31.19:4842 , piece_id=NMEOAL3GYOAAQFFTN7V7MFBD5D6DBZ2A2ZKAFYDNSX5U6TWDIBWA, seg_pos={0 0}, upload=359010]: Done uploading piece. err: <nil> Dec 22 16:25:43.430530702 (408h56m43.301613737s) [noise=true , address=45.91.116.12:4225 , piece_id=2DA5MBVLDWJHJHQUB6V7HMEMMIANHLPIKCGMXEO22DZ632MRGRXQ, seg_pos={0 0}, upload=359010]: Done uploading piece. err: <nil> Dec 22 16:25:43.607233011 (408h56m43.478316046s) [noise=true , address=145.79.227.45:4229 , piece_id=LHMUNX6RFSMINZBT273YCG5DXXJDKZL6O4PHBP3CUZQGVSXI3ZKA, seg_pos={0 0}, upload=359010]: Done uploading piece. err: <nil> Dec 22 16:25:43.615452442 (408h56m43.486535487s) [noise=true , address=145.79.227.35:4481 , piece_id=P2RXVG2UOSBVDPI3RM7JQI3AVOSGVGG7SJ3B2AM2WCJZATT2YAWA, seg_pos={0 0}, upload=359010]: Done uploading piece. err: <nil> Dec 22 16:25:43.616375138 (408h56m43.487458173s) [noise=true , address=145.79.227.31:4799 , piece_id=KMPAWOZTTYLUAVFQ4FO5XL6J2WV6LTYMKQ4NN2JOQHFJ7PHJDNMQ, seg_pos={0 0}, upload=359010]: Done uploading piece. err: <nil> Dec 22 16:25:43.625864228 (408h56m43.496947273s) [noise=true , address=145.79.227.46:4603 , piece_id=XXP4HUASFRX6TVFBTXMUHEU5I77YAXOEMCBI3N5NH2TSMAIOMN5A, seg_pos={0 0}, upload=359010]: Done uploading piece. err: <nil> Dec 22 16:25:43.633964582 (408h56m43.505047617s) [noise=true , address=145.79.227.37:4565 , piece_id=CGUBWJTUNAHCPSFDOJ4SK4ESMDQBLXCBSXG64GL4GTG35NS2FNDQ, seg_pos={0 0}, upload=359010]: Done uploading piece. err: <nil> Dec 22 16:25:43.634530840 (408h56m43.505613875s) [noise=true , address=145.79.227.43:4157 , piece_id=YDS6YME6V37C55NLNH4P3YW6GMF5UFBJTP33GVTLWQ5MBXTEZTFQ, seg_pos={0 0}, upload=359010]: Done uploading piece. err: <nil> Dec 22 16:25:43.656709091 (408h56m43.527792126s) [noise=true , address=145.79.227.51:4411 , piece_id=ICNWN2QTSMN7HEBL3NKB3US4Q6AH6RZZVLMEC6R7VSVV4P3UYHJQ, seg_pos={0 0}, upload=359010]: Done uploading piece. err: <nil> Dec 22 16:25:43.677506783 (408h56m43.548589818s) [noise=true , address=45.91.116.9:4264 , piece_id=IBQEIAODVZGVHWPYB6SZ2WLJC5GBU2VFDMGRFZI2WFGYNKQQJ2HQ, seg_pos={0 0}, upload=359010]: Done uploading piece. err: <nil> Dec 22 16:25:43.867878541 (408h56m43.738961586s) [noise=true , address=145.79.227.19:4078 , piece_id=DPQ4PUJK7AJPDT5HNTKIL3RXFF3357TT7BT7B4YQKBAX52T2MRBA, seg_pos={0 0}, upload=359010]: Done uploading piece. err: <nil> Dec 22 16:25:43.868572843 (408h56m43.739655878s) [noise=true , address=145.79.227.11:4732 , piece_id=D4IPGGRMQB2HZJUNVCSCV5KCD2VH2FE72B4FUNNM56JU2WMQ3CXA, seg_pos={0 0}, upload=359010]: Done uploading piece. err: <nil> Dec 22 16:25:43.868584325 (408h56m43.73966737s) [seg_pos={0 0}, upload=359010]: Segment reached optimal threshold of 13 pieces. Dec 22 16:25:43.868820244 (408h56m43.739903279s) [noise=true , address=145.79.227.10:4401 , piece_id=RNLQMUK2ATVQXAKHVRF4KHX4ENQWCOYICYXYLPAGZUK2NXTWCW5Q, seg_pos={0 0}, upload=359010]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1): protocol: expected piece hash; context canceled; EOF Dec 22 16:25:43.868829802 (408h56m43.739912847s) [noise=true , address=145.79.227.10:4401 , piece_id=RNLQMUK2ATVQXAKHVRF4KHX4ENQWCOYICYXYLPAGZUK2NXTWCW5Q, seg_pos={0 0}, upload=359010]: Long tail cancellation due to optimal threshold Dec 22 16:25:43.869117540 (408h56m43.740200585s) [noise=true , address=45.91.116.11:4118 , piece_id=J2G4RBUUKNZ2H7TYJ234OTM2PECY66TRFEBRTJ6NQC2RT2BOEKFQ, seg_pos={0 0}, upload=359010]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Dec 22 16:25:43.869126488 (408h56m43.740209533s) [noise=true , address=45.91.116.11:4118 , piece_id=J2G4RBUUKNZ2H7TYJ234OTM2PECY66TRFEBRTJ6NQC2RT2BOEKFQ, seg_pos={0 0}, upload=359010]: Long tail cancellation due to optimal threshold Dec 22 16:25:43.869300800 (408h56m43.740383835s) [noise=true , address=145.79.227.49:4338 , piece_id=KAEYGZFDIW2L4YRUOW5XCCIWFTISQA4ORGE3UHO3G55YTB4P54UQ, seg_pos={0 0}, upload=359010]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN): protocol: expected piece hash; context canceled; EOF Dec 22 16:25:43.869308975 (408h56m43.74039202s) [noise=true , address=145.79.227.49:4338 , piece_id=KAEYGZFDIW2L4YRUOW5XCCIWFTISQA4ORGE3UHO3G55YTB4P54UQ, seg_pos={0 0}, upload=359010]: Long tail cancellation due to optimal threshold Dec 22 16:25:43.869327901 (408h56m43.740410957s) [seg_pos={0 0}, upload=359010]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 22 16:25:43.869403946 (408h56m43.740486991s) [upload=359010]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 16:25:43.869410508 (408h56m43.740493553s) [upload=359010]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 16:25:44.290535067 (408h56m44.161618172s) [upload=359010]: Done uploading segments... Dec 22 16:25:44.290561687 (408h56m44.161644762s) [upload=359010]: Upload finished. err: <nil> Duration: 30.004254573s core uplink us-select-1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 497.747891ms target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/704b2d4a066e7708 FAIL (2025-12-22 16:26:03 UTC) orbiter warning: Get "https://eu1.storj.io/login": dial tcp 35.207.121.91:443: i/o timeout Duration: 497.877918ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [X...X..XXXX....X..X...XX.....XXXX......X.X.X.X...X....XXXXXX] duration: 30.002808198s trace: https://jaeger.dev.storj.io/trace/7d766768a0d9de02 FAIL (2025-12-22 16:25:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 22 16:25:10.207471284 (408h56m10.078554319s) [upload=359004]: Starting upload Dec 22 16:25:10.207524345 (408h56m10.07860739s) [upload=359004]: Done starting upload Dec 22 16:25:10.207619877 (408h56m10.078702912s) [upload=359004]: Uploading segments... Dec 22 16:25:10.207840557 (408h56m10.078923592s) [upload=359004]: Got next segment. Inline: false Dec 22 16:25:10.207877528 (408h56m10.078960573s) [upload=359004]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 16:25:10.207882978 (408h56m10.078966013s) [upload=359004]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 16:25:10.453605592 (408h56m10.324688637s) [seg_pos={0 0}, upload=359004]: Begin upload segment... Dec 22 16:25:10.453654937 (408h56m10.324737982s) [seg_pos={0 0}, upload=359004]: Done begin upload segment. Dec 22 16:25:10.453686266 (408h56m10.324769311s) [noise=true , address=77.90.31.16:4736 , piece_id=RRHVHTSYJ3VN5TQOT3HKMFBCBYO4RQ3MASEI42D77QVUYVELPZNQ, seg_pos={0 0}, upload=359004]: Uploading piece... Dec 22 16:25:10.453754916 (408h56m10.324837951s) [noise=true , address=145.79.227.48:4301 , piece_id=A42EUYIUE4O6WGDT3FEKTT4M5SQKAJX3BWBRXVAPGXO2BKCUBRYQ, seg_pos={0 0}, upload=359004]: Uploading piece... Dec 22 16:25:10.454484716 (408h56m10.325567761s) [noise=true , address=145.79.227.29:4943 , piece_id=QVXAJ7S7AWCJH25NAQHEKZ5X2TMGJK2XIQOHJZNZ25LXNLGNCWHA, seg_pos={0 0}, upload=359004]: Uploading piece... Dec 22 16:25:10.454902241 (408h56m10.325985276s) [noise=true , address=145.79.227.46:4602 , piece_id=ZVAIUZ3YJWJATVQ2V7VFPAEKBFHW4T7VLBGNWMTD6VG2Y7QZ6YLA, seg_pos={0 0}, upload=359004]: Uploading piece... Dec 22 16:25:10.455292094 (408h56m10.326375139s) [noise=true , address=45.91.116.4:4939 , piece_id=BK6IQ7T4FT5ENVRPST6UMM4ZK5MVS4KOVKWHRFQ5DDBHZVZ26NJQ, seg_pos={0 0}, upload=359004]: Uploading piece... Dec 22 16:25:10.455787577 (408h56m10.326870612s) [noise=true , address=45.91.116.8:4045 , piece_id=MPKNOBXQX6O2QIDJXNDTCRFPYPGGI5BNPBGDRUN4JGVAX73QGWSA, seg_pos={0 0}, upload=359004]: Uploading piece... Dec 22 16:25:10.456280526 (408h56m10.327363571s) [noise=true , address=145.79.227.45:4229 , piece_id=NBCWCIFXQL4HBVGYI4542UVFZJU5OXUFSLVWIE5RLJPUFCOBI62A, seg_pos={0 0}, upload=359004]: Uploading piece... Dec 22 16:25:10.456762584 (408h56m10.327845619s) [noise=true , address=145.79.227.37:4566 , piece_id=7R7W62EYRWI6BFXMHOJJOH24LPVVK5V6U24GWNPOQKRRUM53USLA, seg_pos={0 0}, upload=359004]: Uploading piece... Dec 22 16:25:10.458099018 (408h56m10.329182063s) [noise=true , address=45.91.116.14:4192 , piece_id=ORDVCAEY2T7TTTBSAALBEOU36BHZROORJNSWLBUXMN5LBAKMAWPQ, seg_pos={0 0}, upload=359004]: Uploading piece... Dec 22 16:25:10.458263572 (408h56m10.329346617s) [noise=true , address=45.91.116.7:4010 , piece_id=A36RYSOIOX2BRV6AX776CQN6XWSJZ3I4SFCON2RMOFLCUHJ3N3AA, seg_pos={0 0}, upload=359004]: Uploading piece... Dec 22 16:25:10.459645764 (408h56m10.330728809s) [noise=true , address=145.79.227.42:4121 , piece_id=EKCIWLN7EB7R4IOSVMBGNNUSOQWOTM54B5INFSGRXUQS6G2YQJVA, seg_pos={0 0}, upload=359004]: Uploading piece... Dec 22 16:25:10.462123302 (408h56m10.333206337s) [noise=true , address=145.79.227.14:4837 , piece_id=DI5YZZVEOWSMW5AEOEQ5JZA4EQLA3UVBNZ3Y7UEPIA33FWDQGSAA, seg_pos={0 0}, upload=359004]: Uploading piece... Dec 22 16:25:10.464098483 (408h56m10.335181518s) [noise=true , address=77.90.31.14:4662 , piece_id=3AFL3D6WPHNOHOCB6MBZENWIS56LBVUXZ53MDDCQ77IYD7FS23DA, seg_pos={0 0}, upload=359004]: Uploading piece... Dec 22 16:25:10.465633807 (408h56m10.336716842s) [noise=true , address=77.90.31.11:4556 , piece_id=MLBAOUPCWIDPWETMA6CJBWVR2ZRJXJJA3SHVM3SUTH6264K4ZCLA, seg_pos={0 0}, upload=359004]: Uploading piece... Dec 22 16:25:10.466398342 (408h56m10.337481387s) [noise=true , address=45.91.116.9:4261 , piece_id=UWGV326SMJLSJZ3PSOQJ2BWRFJZIREYVTDGFLGGZF5OXIBGUW3ZQ, seg_pos={0 0}, upload=359004]: Uploading piece... Dec 22 16:25:10.467830289 (408h56m10.338913324s) [noise=true , address=145.79.227.36:4518 , piece_id=HBQDT5VMAXJO5B5V6HSSXE5BJECBPPZ7WXN3CUM33Y2CV743ZBAA, seg_pos={0 0}, upload=359004]: Uploading piece... Dec 22 16:25:12.319519666 (408h56m12.190602701s) [upload=359004]: Next returned nil segment Dec 22 16:25:12.319539174 (408h56m12.190622219s) [upload=359004]: Waiting for error group managing segments... Dec 22 16:25:12.726707226 (408h56m12.597790261s) [noise=true , address=45.91.116.4:4939 , piece_id=BK6IQ7T4FT5ENVRPST6UMM4ZK5MVS4KOVKWHRFQ5DDBHZVZ26NJQ, seg_pos={0 0}, upload=359004]: Done uploading piece. err: <nil> Dec 22 16:25:12.745140857 (408h56m12.616223902s) [noise=true , address=77.90.31.16:4736 , piece_id=RRHVHTSYJ3VN5TQOT3HKMFBCBYO4RQ3MASEI42D77QVUYVELPZNQ, seg_pos={0 0}, upload=359004]: Done uploading piece. err: <nil> Dec 22 16:25:12.760977512 (408h56m12.632060547s) [noise=true , address=45.91.116.8:4045 , piece_id=MPKNOBXQX6O2QIDJXNDTCRFPYPGGI5BNPBGDRUN4JGVAX73QGWSA, seg_pos={0 0}, upload=359004]: Done uploading piece. err: <nil> Dec 22 16:25:12.767455207 (408h56m12.638538252s) [noise=true , address=45.91.116.14:4192 , piece_id=ORDVCAEY2T7TTTBSAALBEOU36BHZROORJNSWLBUXMN5LBAKMAWPQ, seg_pos={0 0}, upload=359004]: Done uploading piece. err: <nil> Dec 22 16:25:12.770096998 (408h56m12.641180043s) [noise=true , address=45.91.116.7:4010 , piece_id=A36RYSOIOX2BRV6AX776CQN6XWSJZ3I4SFCON2RMOFLCUHJ3N3AA, seg_pos={0 0}, upload=359004]: Done uploading piece. err: <nil> Dec 22 16:25:13.010151188 (408h56m12.881234233s) [noise=true , address=145.79.227.36:4518 , piece_id=HBQDT5VMAXJO5B5V6HSSXE5BJECBPPZ7WXN3CUM33Y2CV743ZBAA, seg_pos={0 0}, upload=359004]: Done uploading piece. err: <nil> Dec 22 16:25:13.029111783 (408h56m12.900194828s) [noise=true , address=145.79.227.14:4837 , piece_id=DI5YZZVEOWSMW5AEOEQ5JZA4EQLA3UVBNZ3Y7UEPIA33FWDQGSAA, seg_pos={0 0}, upload=359004]: Done uploading piece. err: <nil> Dec 22 16:25:13.039272349 (408h56m12.910355394s) [noise=true , address=145.79.227.29:4943 , piece_id=QVXAJ7S7AWCJH25NAQHEKZ5X2TMGJK2XIQOHJZNZ25LXNLGNCWHA, seg_pos={0 0}, upload=359004]: Done uploading piece. err: <nil> Dec 22 16:25:13.202046808 (408h56m13.073129843s) [noise=true , address=145.79.227.48:4301 , piece_id=A42EUYIUE4O6WGDT3FEKTT4M5SQKAJX3BWBRXVAPGXO2BKCUBRYQ, seg_pos={0 0}, upload=359004]: Done uploading piece. err: <nil> Dec 22 16:25:13.216125736 (408h56m13.087208771s) [noise=true , address=145.79.227.42:4121 , piece_id=EKCIWLN7EB7R4IOSVMBGNNUSOQWOTM54B5INFSGRXUQS6G2YQJVA, seg_pos={0 0}, upload=359004]: Done uploading piece. err: <nil> Dec 22 16:25:13.384277834 (408h56m13.25536088s) [noise=true , address=45.91.116.9:4261 , piece_id=UWGV326SMJLSJZ3PSOQJ2BWRFJZIREYVTDGFLGGZF5OXIBGUW3ZQ, seg_pos={0 0}, upload=359004]: Done uploading piece. err: <nil> Dec 22 16:25:13.407211294 (408h56m13.278294329s) [noise=true , address=145.79.227.45:4229 , piece_id=NBCWCIFXQL4HBVGYI4542UVFZJU5OXUFSLVWIE5RLJPUFCOBI62A, seg_pos={0 0}, upload=359004]: Done uploading piece. err: <nil> Dec 22 16:25:13.412855511 (408h56m13.283938546s) [noise=true , address=145.79.227.46:4602 , piece_id=ZVAIUZ3YJWJATVQ2V7VFPAEKBFHW4T7VLBGNWMTD6VG2Y7QZ6YLA, seg_pos={0 0}, upload=359004]: Done uploading piece. err: <nil> Dec 22 16:25:13.412877463 (408h56m13.283960498s) [seg_pos={0 0}, upload=359004]: Segment reached optimal threshold of 13 pieces. Dec 22 16:25:13.413053258 (408h56m13.284136293s) [noise=true , address=77.90.31.14:4662 , piece_id=3AFL3D6WPHNOHOCB6MBZENWIS56LBVUXZ53MDDCQ77IYD7FS23DA, seg_pos={0 0}, upload=359004]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Dec 22 16:25:13.413058929 (408h56m13.284141974s) [noise=true , address=77.90.31.14:4662 , piece_id=3AFL3D6WPHNOHOCB6MBZENWIS56LBVUXZ53MDDCQ77IYD7FS23DA, seg_pos={0 0}, upload=359004]: Long tail cancellation due to optimal threshold Dec 22 16:25:13.413259430 (408h56m13.284342475s) [noise=true , address=77.90.31.11:4556 , piece_id=MLBAOUPCWIDPWETMA6CJBWVR2ZRJXJJA3SHVM3SUTH6264K4ZCLA, seg_pos={0 0}, upload=359004]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks): protocol: expected piece hash; context canceled; EOF Dec 22 16:25:13.413264480 (408h56m13.284347515s) [noise=true , address=77.90.31.11:4556 , piece_id=MLBAOUPCWIDPWETMA6CJBWVR2ZRJXJJA3SHVM3SUTH6264K4ZCLA, seg_pos={0 0}, upload=359004]: Long tail cancellation due to optimal threshold Dec 22 16:25:13.413369150 (408h56m13.284452185s) [noise=true , address=145.79.227.37:4566 , piece_id=7R7W62EYRWI6BFXMHOJJOH24LPVVK5V6U24GWNPOQKRRUM53USLA, seg_pos={0 0}, upload=359004]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SaNJZaVix7BSbEWhu2Vmt3XjDDWCsQ8wTSiFR7MQ8oGwD9N2F): protocol: expected piece hash; context canceled; EOF Dec 22 16:25:13.413374299 (408h56m13.284457334s) [noise=true , address=145.79.227.37:4566 , piece_id=7R7W62EYRWI6BFXMHOJJOH24LPVVK5V6U24GWNPOQKRRUM53USLA, seg_pos={0 0}, upload=359004]: Long tail cancellation due to optimal threshold Dec 22 16:25:13.413391321 (408h56m13.284474356s) [seg_pos={0 0}, upload=359004]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 22 16:25:13.413474721 (408h56m13.284557766s) [upload=359004]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 16:25:13.413480772 (408h56m13.284563827s) [upload=359004]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 16:25:13.701057112 (408h56m13.572140157s) [upload=359004]: Done uploading segments... Dec 22 16:25:13.701078613 (408h56m13.572161658s) [upload=359004]: Upload finished. err: <nil> Duration: 30.002826883s core uplink us-select-1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 478.822664ms target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/704b2d4a066e7708 FAIL (2025-12-22 16:25:17 UTC) orbiter warning: Get "https://eu1.storj.io/login": dial tcp 35.207.121.91:443: i/o timeout Duration: 478.851157ms core satellite eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 1.375383377s trace: https://jaeger.dev.storj.io/trace/6ea3fd74aff47917 FAIL (2025-12-22 16:25:13 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.37541118s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X............................X.......................] duration: 5.85983812s trace: https://jaeger.dev.storj.io/trace/72726d8466c916ea FAIL (2025-12-22 16:25:08 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 5.859875169s core uplink eu1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 468.178937ms target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/704b2d4a066e7708 FAIL (2025-12-22 16:25:01 UTC) orbiter warning: Get "https://eu1.storj.io/login": dial tcp 35.207.121.91:443: i/o timeout Duration: 468.210797ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 24 out of 60 is failed (threshold=3) [.X...X..XXXX....X..X...XX.....XXXX......X.X.X.X...X....XXXXX] duration: 30.003301878s trace: https://jaeger.dev.storj.io/trace/6b11a699df79bb76 FAIL (2025-12-22 16:24:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 22 16:24:17.794512027 (408h55m17.665595062s) [upload=358991]: Starting upload Dec 22 16:24:17.794606137 (408h55m17.665689172s) [upload=358991]: Done starting upload Dec 22 16:24:17.794647105 (408h55m17.66573015s) [upload=358991]: Uploading segments... Dec 22 16:24:17.794963577 (408h55m17.666046632s) [upload=358991]: Got next segment. Inline: false Dec 22 16:24:17.795003263 (408h55m17.666086308s) [upload=358991]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 16:24:17.795008873 (408h55m17.666091928s) [upload=358991]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 16:24:18.036763703 (408h55m17.907846738s) [seg_pos={0 0}, upload=358991]: Begin upload segment... Dec 22 16:24:18.036819218 (408h55m17.907902263s) [seg_pos={0 0}, upload=358991]: Done begin upload segment. Dec 22 16:24:18.036907667 (408h55m17.907990702s) [noise=true , address=45.91.116.13:4153 , piece_id=ZUKCEJFYK7IEIR3SWXBIX32U3DK2QFIFD26IGJBBU2NGJU7QH5JQ, seg_pos={0 0}, upload=358991]: Uploading piece... Dec 22 16:24:18.036931072 (408h55m17.908014107s) [noise=true , address=77.90.31.11:4556 , piece_id=GVMXX7JIWPHYZ4Q7NU6O4GIZ4WSNBO2XMXPEGKAGB7K35YDIKNHA, seg_pos={0 0}, upload=358991]: Uploading piece... Dec 22 16:24:18.037130932 (408h55m17.908213967s) [noise=true , address=145.79.227.45:4229 , piece_id=WG6HBR2VLEF2XYHCFO7EKV3XZPNBXP6NTJBGRRAVBPGDVU74NP2A, seg_pos={0 0}, upload=358991]: Uploading piece... Dec 22 16:24:18.037709284 (408h55m17.908792329s) [noise=true , address=77.90.31.19:4841 , piece_id=A7PWIG2GQR72KHIPJLFJSQB6TYFTPAZRGUMJKPJYD7PJJNPANOEA, seg_pos={0 0}, upload=358991]: Uploading piece... Dec 22 16:24:18.038140645 (408h55m17.90922368s) [noise=true , address=145.79.227.20:4438 , piece_id=4ZC4BDKNEOJGEKV6N5Y7SKA6D5MH37FMX5MNXQ2KBP6SNGVEQVIA, seg_pos={0 0}, upload=358991]: Uploading piece... Dec 22 16:24:18.038263850 (408h55m17.909346885s) [noise=true , address=77.90.31.15:4700 , piece_id=N6DM7CFB744TQYJEGZI7M6EUJSB5QPHY7VZ6XHJA57NXM7C4BFPQ, seg_pos={0 0}, upload=358991]: Uploading piece... Dec 22 16:24:18.038436569 (408h55m17.909519614s) [noise=true , address=145.79.227.29:4941 , piece_id=6A53I7VK6PGYWDCVCUNHSAZKMQDLIOUGGB5BM7AX33U26QKKV3GA, seg_pos={0 0}, upload=358991]: Uploading piece... Dec 22 16:24:18.038787898 (408h55m17.909870943s) [noise=true , address=45.91.116.5:4298 , piece_id=VLYNON5RP3VJZIBAFCO42Z6VOITYK5HA34L5TV7I5Y2VXH3IG5MA, seg_pos={0 0}, upload=358991]: Uploading piece... Dec 22 16:24:18.038882358 (408h55m17.909965403s) [noise=true , address=145.79.227.37:4567 , piece_id=O3SZJQE2N5B2GR4N2PVT2AZUOZW5ZEB6KR6K6URRA6TYOODQZGWA, seg_pos={0 0}, upload=358991]: Uploading piece... Dec 22 16:24:18.039083119 (408h55m17.910166164s) [noise=true , address=45.91.116.6:4973 , piece_id=KZ3N2OYQVLSHGNPA56BBVPSWHG7IOBTWJYOWRLQCQRVLZBGGIANA, seg_pos={0 0}, upload=358991]: Uploading piece... Dec 22 16:24:18.039320552 (408h55m17.910403597s) [noise=true , address=45.91.116.11:4120 , piece_id=XCWT5BRJ2CXQE6EJAGTLLFK4NOIEOYSIUOOPJEVC475YN4EFVPRA, seg_pos={0 0}, upload=358991]: Uploading piece... Dec 22 16:24:18.039381778 (408h55m17.910464813s) [noise=true , address=145.79.227.32:4834 , piece_id=KQ3VMW2HYM524EAV4X4PYS5KXILDFAOFVENJGYD2RUKWLVHJXD4Q, seg_pos={0 0}, upload=358991]: Uploading piece... Dec 22 16:24:18.039837817 (408h55m17.910920862s) [noise=true , address=77.90.31.14:4662 , piece_id=ZEQGSB4AXKVHVLIGT2L2KOQWZ243ZK3FLJ2MRTZRZTIGFSKLAFTA, seg_pos={0 0}, upload=358991]: Uploading piece... Dec 22 16:24:18.040375079 (408h55m17.911458124s) [noise=true , address=45.91.116.7:4010 , piece_id=BCNDVHTN2WJ2O3ZIP7WY4UHKEPMQROO3ZUE5MKOWTKXBFKIYY3XQ, seg_pos={0 0}, upload=358991]: Uploading piece... Dec 22 16:24:18.040901391 (408h55m17.911984436s) [noise=true , address=145.79.227.15:4875 , piece_id=V7RHVYGDMTDHITWYCUI5EDDT3WWNAJV4K6V4CN7SVF3EKJZONUNQ, seg_pos={0 0}, upload=358991]: Uploading piece... Dec 22 16:24:18.042897924 (408h55m17.913980969s) [noise=true , address=145.79.227.43:4159 , piece_id=YWPQNYGIUVMAO2DPNDYBDDYFUYFRKUGVZ4FMMS3GCOAH44NJZROQ, seg_pos={0 0}, upload=358991]: Uploading piece... Dec 22 16:24:19.134050862 (408h55m19.005133907s) [noise=true , address=77.90.31.19:4841 , piece_id=A7PWIG2GQR72KHIPJLFJSQB6TYFTPAZRGUMJKPJYD7PJJNPANOEA, seg_pos={0 0}, upload=358991]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000 Dec 22 16:24:19.944104963 (408h55m19.815187998s) [upload=358991]: Next returned nil segment Dec 22 16:24:19.944121283 (408h55m19.815204328s) [upload=358991]: Waiting for error group managing segments... Dec 22 16:24:20.260377496 (408h55m20.131460541s) [noise=true , address=77.90.31.15:4700 , piece_id=N6DM7CFB744TQYJEGZI7M6EUJSB5QPHY7VZ6XHJA57NXM7C4BFPQ, seg_pos={0 0}, upload=358991]: Done uploading piece. err: <nil> Dec 22 16:24:20.306570424 (408h55m20.177653469s) [noise=true , address=45.91.116.13:4153 , piece_id=ZUKCEJFYK7IEIR3SWXBIX32U3DK2QFIFD26IGJBBU2NGJU7QH5JQ, seg_pos={0 0}, upload=358991]: Done uploading piece. err: <nil> Dec 22 16:24:20.309030529 (408h55m20.180113574s) [noise=true , address=45.91.116.6:4973 , piece_id=KZ3N2OYQVLSHGNPA56BBVPSWHG7IOBTWJYOWRLQCQRVLZBGGIANA, seg_pos={0 0}, upload=358991]: Done uploading piece. err: <nil> Dec 22 16:24:20.332094226 (408h55m20.203177261s) [noise=true , address=45.91.116.11:4120 , piece_id=XCWT5BRJ2CXQE6EJAGTLLFK4NOIEOYSIUOOPJEVC475YN4EFVPRA, seg_pos={0 0}, upload=358991]: Done uploading piece. err: <nil> Dec 22 16:24:20.332717202 (408h55m20.203800237s) [noise=true , address=77.90.31.14:4662 , piece_id=ZEQGSB4AXKVHVLIGT2L2KOQWZ243ZK3FLJ2MRTZRZTIGFSKLAFTA, seg_pos={0 0}, upload=358991]: Done uploading piece. err: <nil> Dec 22 16:24:20.799804800 (408h55m20.670887845s) [noise=true , address=45.91.116.7:4010 , piece_id=BCNDVHTN2WJ2O3ZIP7WY4UHKEPMQROO3ZUE5MKOWTKXBFKIYY3XQ, seg_pos={0 0}, upload=358991]: Done uploading piece. err: <nil> Dec 22 16:24:20.799820962 (408h55m20.670903997s) [noise=true , address=145.79.227.32:4834 , piece_id=KQ3VMW2HYM524EAV4X4PYS5KXILDFAOFVENJGYD2RUKWLVHJXD4Q, seg_pos={0 0}, upload=358991]: Done uploading piece. err: <nil> Dec 22 16:24:20.849140093 (408h55m20.720223138s) [noise=true , address=145.79.227.45:4229 , piece_id=WG6HBR2VLEF2XYHCFO7EKV3XZPNBXP6NTJBGRRAVBPGDVU74NP2A, seg_pos={0 0}, upload=358991]: Done uploading piece. err: <nil> Dec 22 16:24:20.853431454 (408h55m20.724514489s) [noise=true , address=145.79.227.29:4941 , piece_id=6A53I7VK6PGYWDCVCUNHSAZKMQDLIOUGGB5BM7AX33U26QKKV3GA, seg_pos={0 0}, upload=358991]: Done uploading piece. err: <nil> Dec 22 16:24:20.859130056 (408h55m20.730213101s) [noise=true , address=145.79.227.15:4875 , piece_id=V7RHVYGDMTDHITWYCUI5EDDT3WWNAJV4K6V4CN7SVF3EKJZONUNQ, seg_pos={0 0}, upload=358991]: Done uploading piece. err: <nil> Dec 22 16:24:20.859578059 (408h55m20.730661094s) [noise=true , address=145.79.227.43:4159 , piece_id=YWPQNYGIUVMAO2DPNDYBDDYFUYFRKUGVZ4FMMS3GCOAH44NJZROQ, seg_pos={0 0}, upload=358991]: Done uploading piece. err: <nil> Dec 22 16:24:20.872133977 (408h55m20.743217052s) [noise=true , address=145.79.227.20:4438 , piece_id=4ZC4BDKNEOJGEKV6N5Y7SKA6D5MH37FMX5MNXQ2KBP6SNGVEQVIA, seg_pos={0 0}, upload=358991]: Done uploading piece. err: <nil> Dec 22 16:24:21.012139359 (408h55m20.883222404s) [noise=true , address=145.79.227.37:4567 , piece_id=O3SZJQE2N5B2GR4N2PVT2AZUOZW5ZEB6KR6K6URRA6TYOODQZGWA, seg_pos={0 0}, upload=358991]: Done uploading piece. err: <nil> Dec 22 16:24:21.012166161 (408h55m20.883249196s) [seg_pos={0 0}, upload=358991]: Segment reached optimal threshold of 13 pieces. Dec 22 16:24:21.012409494 (408h55m20.883492529s) [noise=true , address=45.91.116.5:4298 , piece_id=VLYNON5RP3VJZIBAFCO42Z6VOITYK5HA34L5TV7I5Y2VXH3IG5MA, seg_pos={0 0}, upload=358991]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Dec 22 16:24:21.012420304 (408h55m20.883503339s) [noise=true , address=45.91.116.5:4298 , piece_id=VLYNON5RP3VJZIBAFCO42Z6VOITYK5HA34L5TV7I5Y2VXH3IG5MA, seg_pos={0 0}, upload=358991]: Long tail cancellation due to optimal threshold Dec 22 16:24:21.012512250 (408h55m20.883595295s) [noise=true , address=77.90.31.11:4556 , piece_id=GVMXX7JIWPHYZ4Q7NU6O4GIZ4WSNBO2XMXPEGKAGB7K35YDIKNHA, seg_pos={0 0}, upload=358991]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks): protocol: expected piece hash; context canceled; EOF Dec 22 16:24:21.012527408 (408h55m20.883610443s) [noise=true , address=77.90.31.11:4556 , piece_id=GVMXX7JIWPHYZ4Q7NU6O4GIZ4WSNBO2XMXPEGKAGB7K35YDIKNHA, seg_pos={0 0}, upload=358991]: Long tail cancellation due to optimal threshold Dec 22 16:24:21.012544952 (408h55m20.883627987s) [seg_pos={0 0}, upload=358991]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 22 16:24:21.012637407 (408h55m20.883720442s) [upload=358991]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 16:24:21.012642437 (408h55m20.883725482s) [upload=358991]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 16:24:21.316395892 (408h55m21.187478937s) [upload=358991]: Done uploading segments... Dec 22 16:24:21.316418836 (408h55m21.187501881s) [upload=358991]: Upload finished. err: <nil> Duration: 30.003312449s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 1.382236989s trace: https://jaeger.dev.storj.io/trace/6ea3fd74aff47917 FAIL (2025-12-22 16:24:34 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.382261737s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........X............................X......................] duration: 5.47624053s trace: https://jaeger.dev.storj.io/trace/72726d8466c916ea FAIL (2025-12-22 16:24:08 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 5.476272261s core uplink eu1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 469.11507ms target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/704b2d4a066e7708 FAIL (2025-12-22 16:23:59 UTC) orbiter warning: Get "https://eu1.storj.io/login": dial tcp 35.207.121.91:443: i/o timeout Duration: 469.145158ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [..X...X..XXXX....X..X...XX.....XXXX......X.X.X.X...X....XXXX] duration: 30.002096554s trace: https://jaeger.dev.storj.io/trace/1934502f109f4b65 FAIL (2025-12-22 16:23:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, Piece ID: CKZJTIG7L4YFSPDQXLZMJHXEN7IWPHYPXX4BFI3N32V2J6YGWTJQ) context deadline exceeded; piecestore: (Node ID: 1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, Piece ID: QA67JZHEMTRVQGAT2MQRLZIJOJX4CZTFCLYRKDKXWSOSXSTAYQHA) context deadline exceeded; piecestore: (Node ID: 12FZuwK69Hju5JKoujYS98XohYQgcfm3uea1Wk9bcsMzPW7hXLs, Piece ID: YV2KEFSRZSNGEZ6Y2ECX4WEQJ3K36YIEN4JTZWRPTAO73K5QFSZQ) context deadline exceeded ======LOGS====== Dec 22 16:23:20.353080574 (408h54m20.224163609s) [upload=358977]: Starting upload Dec 22 16:23:20.353164854 (408h54m20.224247889s) [upload=358977]: Done starting upload Dec 22 16:23:20.353336901 (408h54m20.224419936s) [upload=358977]: Uploading segments... Dec 22 16:23:20.353384212 (408h54m20.224467267s) [upload=358977]: Got next segment. Inline: false Dec 22 16:23:20.353411423 (408h54m20.224494458s) [upload=358977]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 16:23:20.353415973 (408h54m20.224499008s) [upload=358977]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 16:23:20.594131863 (408h54m20.465214898s) [seg_pos={0 0}, upload=358977]: Begin upload segment... Dec 22 16:23:20.594174215 (408h54m20.46525725s) [seg_pos={0 0}, upload=358977]: Done begin upload segment. Dec 22 16:23:20.594193701 (408h54m20.465276736s) [noise=true , address=145.79.227.50:4373 , piece_id=SZBAUIECYJOGHA2ORC4DOMXTOLNYSLZ23DCMMKYVQFZ7HAVTAZDA, seg_pos={0 0}, upload=358977]: Uploading piece... Dec 22 16:23:20.594218028 (408h54m20.465301093s) [noise=true , address=145.79.227.32:4836 , piece_id=LITVUXPZL7HARFHME7LLTPUZFIFDB3FPDP6ACGFFVUBNPGULPYNA, seg_pos={0 0}, upload=358977]: Uploading piece... Dec 22 16:23:20.594272541 (408h54m20.465355576s) [noise=true , address=145.79.227.39:4013 , piece_id=LKMLKE6Z5EIJ7VRSWOCYU7T2BKFJLRQC3QI267OKGNCZ7VTVZ5UQ, seg_pos={0 0}, upload=358977]: Uploading piece... Dec 22 16:23:20.595323754 (408h54m20.466406789s) [noise=true , address=145.79.227.30:4762 , piece_id=J7WG5RTYIG5N6H2ESZ53AQJW4NKM25BUJG275JK5KTT4LFABAATA, seg_pos={0 0}, upload=358977]: Uploading piece... Dec 22 16:23:20.595955587 (408h54m20.467038622s) [noise=true , address=77.90.31.11:4556 , piece_id=6UZKZ4NHBKCOCBC6VZSKG6JSC42UQG35IDPNANI4GPEBJPWE4HPQ, seg_pos={0 0}, upload=358977]: Uploading piece... Dec 22 16:23:20.596263273 (408h54m20.467346318s) [noise=true , address=45.91.116.7:4012 , piece_id=QA67JZHEMTRVQGAT2MQRLZIJOJX4CZTFCLYRKDKXWSOSXSTAYQHA, seg_pos={0 0}, upload=358977]: Uploading piece... Dec 22 16:23:20.596542164 (408h54m20.467625199s) [noise=true , address=145.79.227.29:4941 , piece_id=EG2FFPZLS2Q7DIXNCIFL52WNPTYSPB2MEB7JG5GUL34RFWOSCVFA, seg_pos={0 0}, upload=358977]: Uploading piece... Dec 22 16:23:20.596821526 (408h54m20.467904561s) [noise=true , address=45.91.116.14:4192 , piece_id=YV2KEFSRZSNGEZ6Y2ECX4WEQJ3K36YIEN4JTZWRPTAO73K5QFSZQ, seg_pos={0 0}, upload=358977]: Uploading piece... Dec 22 16:23:20.597295167 (408h54m20.468378212s) [noise=true , address=145.79.227.44:4194 , piece_id=TKECRAIZOHGSIMRFLYU2GLPDW2C3NR7JHANBQFEWKZUVDB3A6SLA, seg_pos={0 0}, upload=358977]: Uploading piece... Dec 22 16:23:20.598566739 (408h54m20.469649774s) [noise=true , address=145.79.227.45:4229 , piece_id=I6IU6MDLETV67IUQTNJL3QFPR3CGUI3FKKBDH2HKAISOFW4HCDHA, seg_pos={0 0}, upload=358977]: Uploading piece... Dec 22 16:23:20.599180487 (408h54m20.470263522s) [noise=true , address=145.79.227.48:4303 , piece_id=JQ7QORWDV37Q3LUV3CN4XABZYZ4TCP44YJHJ6FSF56LSSB2TZ55A, seg_pos={0 0}, upload=358977]: Uploading piece... Dec 22 16:23:20.600064911 (408h54m20.471147946s) [noise=true , address=145.79.227.36:4519 , piece_id=PJS72CLQLO5MLOEPHAIHXO7LJPHQD5DEAI7TLW7CT2J54FWWM4SA, seg_pos={0 0}, upload=358977]: Uploading piece... Dec 22 16:23:20.600674021 (408h54m20.471757056s) [noise=true , address=145.79.227.33:4870 , piece_id=3BXSLLVEBMKV7V6R6B2Z5ZKM6HNLCW325YUAQA6AK7H6AMT77OZA, seg_pos={0 0}, upload=358977]: Uploading piece... Dec 22 16:23:20.601502418 (408h54m20.472585453s) [noise=true , address=145.79.227.41:4085 , piece_id=BTTPTNL3ZYPKCOXZI3Q75W6VTRXFLMVU4QZQR2255QCF7ZAN23BA, seg_pos={0 0}, upload=358977]: Uploading piece... Dec 22 16:23:20.602143388 (408h54m20.473226423s) [noise=true , address=145.79.227.43:4159 , piece_id=TQKP2PD3PW7F3M7KFGAAHQLOFP4LKHJFZQHJWYPPTIEUFJJJJPFQ, seg_pos={0 0}, upload=358977]: Uploading piece... Dec 22 16:23:20.603510382 (408h54m20.474593417s) [noise=true , address=45.91.116.13:4155 , piece_id=CKZJTIG7L4YFSPDQXLZMJHXEN7IWPHYPXX4BFI3N32V2J6YGWTJQ, seg_pos={0 0}, upload=358977]: Uploading piece... Dec 22 16:23:22.487892053 (408h54m22.358975098s) [upload=358977]: Next returned nil segment Dec 22 16:23:22.487905598 (408h54m22.358988643s) [upload=358977]: Waiting for error group managing segments... Dec 22 16:23:22.835663956 (408h54m22.706747001s) [noise=true , address=45.91.116.14:4192 , piece_id=YV2KEFSRZSNGEZ6Y2ECX4WEQJ3K36YIEN4JTZWRPTAO73K5QFSZQ, seg_pos={0 0}, upload=358977]: Done uploading piece. err: <nil> Dec 22 16:23:22.840784687 (408h54m22.711867732s) [noise=true , address=77.90.31.11:4556 , piece_id=6UZKZ4NHBKCOCBC6VZSKG6JSC42UQG35IDPNANI4GPEBJPWE4HPQ, seg_pos={0 0}, upload=358977]: Done uploading piece. err: <nil> Dec 22 16:23:22.858477226 (408h54m22.729560272s) [noise=true , address=45.91.116.13:4155 , piece_id=CKZJTIG7L4YFSPDQXLZMJHXEN7IWPHYPXX4BFI3N32V2J6YGWTJQ, seg_pos={0 0}, upload=358977]: Done uploading piece. err: <nil> Dec 22 16:23:23.262451658 (408h54m23.133534703s) [noise=true , address=145.79.227.39:4013 , piece_id=LKMLKE6Z5EIJ7VRSWOCYU7T2BKFJLRQC3QI267OKGNCZ7VTVZ5UQ, seg_pos={0 0}, upload=358977]: Done uploading piece. err: <nil> Dec 22 16:23:23.269950957 (408h54m23.141034002s) [noise=true , address=145.79.227.32:4836 , piece_id=LITVUXPZL7HARFHME7LLTPUZFIFDB3FPDP6ACGFFVUBNPGULPYNA, seg_pos={0 0}, upload=358977]: Done uploading piece. err: <nil> Dec 22 16:23:23.290509073 (408h54m23.161592128s) [noise=true , address=145.79.227.36:4519 , piece_id=PJS72CLQLO5MLOEPHAIHXO7LJPHQD5DEAI7TLW7CT2J54FWWM4SA, seg_pos={0 0}, upload=358977]: Done uploading piece. err: <nil> Dec 22 16:23:23.318199520 (408h54m23.189282565s) [noise=true , address=145.79.227.50:4373 , piece_id=SZBAUIECYJOGHA2ORC4DOMXTOLNYSLZ23DCMMKYVQFZ7HAVTAZDA, seg_pos={0 0}, upload=358977]: Done uploading piece. err: <nil> Dec 22 16:23:23.372032406 (408h54m23.243115441s) [noise=true , address=45.91.116.7:4012 , piece_id=QA67JZHEMTRVQGAT2MQRLZIJOJX4CZTFCLYRKDKXWSOSXSTAYQHA, seg_pos={0 0}, upload=358977]: Done uploading piece. err: <nil> Dec 22 16:23:23.516473145 (408h54m23.38755618s) [noise=true , address=145.79.227.45:4229 , piece_id=I6IU6MDLETV67IUQTNJL3QFPR3CGUI3FKKBDH2HKAISOFW4HCDHA, seg_pos={0 0}, upload=358977]: Done uploading piece. err: <nil> Dec 22 16:23:23.539379231 (408h54m23.410462276s) [noise=true , address=145.79.227.48:4303 , piece_id=JQ7QORWDV37Q3LUV3CN4XABZYZ4TCP44YJHJ6FSF56LSSB2TZ55A, seg_pos={0 0}, upload=358977]: Done uploading piece. err: <nil> Dec 22 16:23:23.539464304 (408h54m23.410547349s) [noise=true , address=145.79.227.29:4941 , piece_id=EG2FFPZLS2Q7DIXNCIFL52WNPTYSPB2MEB7JG5GUL34RFWOSCVFA, seg_pos={0 0}, upload=358977]: Done uploading piece. err: <nil> Dec 22 16:23:23.540618512 (408h54m23.411701547s) [noise=true , address=145.79.227.41:4085 , piece_id=BTTPTNL3ZYPKCOXZI3Q75W6VTRXFLMVU4QZQR2255QCF7ZAN23BA, seg_pos={0 0}, upload=358977]: Done uploading piece. err: <nil> Dec 22 16:23:23.548560254 (408h54m23.419643299s) [noise=true , address=145.79.227.43:4159 , piece_id=TQKP2PD3PW7F3M7KFGAAHQLOFP4LKHJFZQHJWYPPTIEUFJJJJPFQ, seg_pos={0 0}, upload=358977]: Done uploading piece. err: <nil> Dec 22 16:23:23.548581764 (408h54m23.419664799s) [seg_pos={0 0}, upload=358977]: Segment reached optimal threshold of 13 pieces. Dec 22 16:23:23.548841279 (408h54m23.419924314s) [noise=true , address=145.79.227.33:4870 , piece_id=3BXSLLVEBMKV7V6R6B2Z5ZKM6HNLCW325YUAQA6AK7H6AMT77OZA, seg_pos={0 0}, upload=358977]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sDH7ccMQhjePqpbTkkxMAtumNLxpMHiT56cnfaYrUTFYxuPVt): protocol: expected piece hash; context canceled; EOF Dec 22 16:23:23.548851378 (408h54m23.419934413s) [noise=true , address=145.79.227.33:4870 , piece_id=3BXSLLVEBMKV7V6R6B2Z5ZKM6HNLCW325YUAQA6AK7H6AMT77OZA, seg_pos={0 0}, upload=358977]: Long tail cancellation due to optimal threshold Dec 22 16:23:23.548970545 (408h54m23.42005359s) [noise=true , address=145.79.227.44:4194 , piece_id=TKECRAIZOHGSIMRFLYU2GLPDW2C3NR7JHANBQFEWKZUVDB3A6SLA, seg_pos={0 0}, upload=358977]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kta1F6RNDP9GJdkiUG3x1Nk6juWJ2tmSDr599DYomSq9C1BBh): protocol: expected piece hash; context canceled; EOF Dec 22 16:23:23.548998919 (408h54m23.420081964s) [noise=true , address=145.79.227.44:4194 , piece_id=TKECRAIZOHGSIMRFLYU2GLPDW2C3NR7JHANBQFEWKZUVDB3A6SLA, seg_pos={0 0}, upload=358977]: Long tail cancellation due to optimal threshold Dec 22 16:23:23.549187819 (408h54m23.420270864s) [noise=true , address=145.79.227.30:4762 , piece_id=J7WG5RTYIG5N6H2ESZ53AQJW4NKM25BUJG275JK5KTT4LFABAATA, seg_pos={0 0}, upload=358977]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1No1jHEK6YFSye92GGBri48Yv2DYUiESY6Gs3Z9PENjmBynS9b): protocol: expected piece hash; context canceled; EOF Dec 22 16:23:23.549196285 (408h54m23.42027933s) [noise=true , address=145.79.227.30:4762 , piece_id=J7WG5RTYIG5N6H2ESZ53AQJW4NKM25BUJG275JK5KTT4LFABAATA, seg_pos={0 0}, upload=358977]: Long tail cancellation due to optimal threshold Dec 22 16:23:23.549218287 (408h54m23.420301332s) [seg_pos={0 0}, upload=358977]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 22 16:23:23.549332364 (408h54m23.420415409s) [upload=358977]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 16:23:23.549338936 (408h54m23.420421971s) [upload=358977]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 16:23:23.836109523 (408h54m23.707192568s) [upload=358977]: Done uploading segments... Dec 22 16:23:23.836128238 (408h54m23.707211273s) [upload=358977]: Upload finished. err: <nil> Duration: 30.002107365s core uplink us-select-1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 465.674438ms target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/704b2d4a066e7708 FAIL (2025-12-22 16:23:38 UTC) orbiter warning: Get "https://eu1.storj.io/login": dial tcp 35.207.121.91:443: i/o timeout Duration: 465.699545ms core satellite eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........X............................X.....................] duration: 6.023607874s trace: https://jaeger.dev.storj.io/trace/72726d8466c916ea FAIL (2025-12-22 16:23:33 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.023624796s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 1.454216459s trace: https://jaeger.dev.storj.io/trace/6ea3fd74aff47917 FAIL (2025-12-22 16:23:29 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.454239262s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [X..X...X..XXXX....X..X...XX.....XXXX......X.X.X.X...X....XXX] duration: 30.00242967s trace: https://jaeger.dev.storj.io/trace/14b01d1645c7d270 FAIL (2025-12-22 16:23:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 22 16:22:50.130649071 (408h53m50.001732116s) [upload=358970]: Starting upload Dec 22 16:22:50.130704627 (408h53m50.001787662s) [upload=358970]: Done starting upload Dec 22 16:22:50.130887565 (408h53m50.0019706s) [upload=358970]: Uploading segments... Dec 22 16:22:50.131004367 (408h53m50.002087413s) [upload=358970]: Got next segment. Inline: false Dec 22 16:22:50.131051758 (408h53m50.002134793s) [upload=358970]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 16:22:50.131057459 (408h53m50.002140504s) [upload=358970]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 16:22:50.380148452 (408h53m50.251231487s) [seg_pos={0 0}, upload=358970]: Begin upload segment... Dec 22 16:22:50.380243733 (408h53m50.251326768s) [seg_pos={0 0}, upload=358970]: Done begin upload segment. Dec 22 16:22:50.380296343 (408h53m50.251379378s) [noise=true , address=145.79.227.48:4303 , piece_id=43EJSUP74OKJFIKQIUZ3RIURVUZAA33GQW6HXCIB5S4BGC3PL5CQ, seg_pos={0 0}, upload=358970]: Uploading piece... Dec 22 16:22:50.380954456 (408h53m50.252037491s) [noise=true , address=145.79.227.33:4869 , piece_id=KXXQEUEQIQFSAQ6ORHLFEOSH44FKL5OXAIEZYCKJG6WFGVE5E3LQ, seg_pos={0 0}, upload=358970]: Uploading piece... Dec 22 16:22:50.380991577 (408h53m50.252074612s) [noise=true , address=145.79.227.13:4803 , piece_id=COEHKUJEXAIDGTEZ2UKUUCO3PHKIQYYGUJKYL7C7BNMIAKJ2477A, seg_pos={0 0}, upload=358970]: Uploading piece... Dec 22 16:22:50.381699795 (408h53m50.25278283s) [noise=true , address=145.79.227.45:4230 , piece_id=7WS62URQHMWTIJ2IEOZ4LZ3GCPAXEEPFFC62LKK6Z3RSWGUHHBEA, seg_pos={0 0}, upload=358970]: Uploading piece... Dec 22 16:22:50.381995168 (408h53m50.253078203s) [noise=true , address=145.79.227.46:4601 , piece_id=52WOVLSR24N2W62HMKLSASXNSPJJBFZRDW6WGRZCAUC56KRI36RA, seg_pos={0 0}, upload=358970]: Uploading piece... Dec 22 16:22:50.382030045 (408h53m50.25311308s) [noise=true , address=145.79.227.29:4942 , piece_id=EJNAXWNS7EEKOQ6EW3ZPILC5CMCXF5JCZFQCFBIR2SRYMO6LMCBQ, seg_pos={0 0}, upload=358970]: Uploading piece... Dec 22 16:22:50.382286092 (408h53m50.253369127s) [noise=true , address=145.79.227.52:4446 , piece_id=ABAYZHL5LDF2GTAS2XGQ7N3DCYOUQKQUC6PFHT2PRCXSKZIYASKA, seg_pos={0 0}, upload=358970]: Uploading piece... Dec 22 16:22:50.382340876 (408h53m50.253423912s) [noise=true , address=145.79.227.9:4366 , piece_id=A5FEG74HZNCBYQK3EDA776ZJB4VTGV6ILQUQ6TBVNSPGX4LANSLA, seg_pos={0 0}, upload=358970]: Uploading piece... Dec 22 16:22:50.382528293 (408h53m50.253611328s) [noise=true , address=145.79.227.17:4972 , piece_id=76V2OUI774LVGW7NSI6JIUIA457VLL5NFM77DW34QGHKI25SXF3Q, seg_pos={0 0}, upload=358970]: Uploading piece... Dec 22 16:22:50.382669542 (408h53m50.253752587s) [noise=true , address=145.79.227.49:4339 , piece_id=CRXDJO422BRXC2DSXFAQCSSFLKBTV7UZLKVKXFPI7YPLYLJHFHHA, seg_pos={0 0}, upload=358970]: Uploading piece... Dec 22 16:22:50.382786175 (408h53m50.25386921s) [noise=true , address=145.79.227.30:4762 , piece_id=L6WJWMMF3G4VBFQ5BKZCDCGOJNEN4DKBDS3YSRQO76CKBITKLIVA, seg_pos={0 0}, upload=358970]: Uploading piece... Dec 22 16:22:50.382944706 (408h53m50.254027741s) [noise=true , address=77.90.31.11:4556 , piece_id=2SWVRRAF4OWRVFFNDPID5H4M7R4B5YSO7INXTLRAQLQUR4EYZXKQ, seg_pos={0 0}, upload=358970]: Uploading piece... Dec 22 16:22:50.383045728 (408h53m50.254128773s) [noise=true , address=45.91.116.6:4976 , piece_id=MI23FFRD5KRUMHI7WDNJMHQAE7KYHHZC5DWUKJXSS4FKLAD7JRGA, seg_pos={0 0}, upload=358970]: Uploading piece... Dec 22 16:22:50.383248545 (408h53m50.25433158s) [noise=true , address=145.79.227.36:4519 , piece_id=RII4MLUE34LSXKEMLCTWG2PLEALFRSDBRRPMM2XWN33EQMQKRHLQ, seg_pos={0 0}, upload=358970]: Uploading piece... Dec 22 16:22:50.383629220 (408h53m50.254712265s) [noise=true , address=45.91.116.12:4225 , piece_id=NB5OOJH5IW7SYGXUGJ2ELTS5NU5G7HXM6CREXCVHWU7TROQKK2LA, seg_pos={0 0}, upload=358970]: Uploading piece... Dec 22 16:22:50.384115586 (408h53m50.255198621s) [noise=true , address=45.91.116.14:4189 , piece_id=43AR2356XPABF7ZR2547YV2A45S3WNQKFLLSXBAKWAASJXRHWUAQ, seg_pos={0 0}, upload=358970]: Uploading piece... Dec 22 16:22:52.292341136 (408h53m52.163424181s) [upload=358970]: Next returned nil segment Dec 22 16:22:52.292353980 (408h53m52.163437015s) [upload=358970]: Waiting for error group managing segments... Dec 22 16:22:52.654471785 (408h53m52.52555482s) [noise=true , address=77.90.31.11:4556 , piece_id=2SWVRRAF4OWRVFFNDPID5H4M7R4B5YSO7INXTLRAQLQUR4EYZXKQ, seg_pos={0 0}, upload=358970]: Done uploading piece. err: <nil> Dec 22 16:22:52.657737133 (408h53m52.528820178s) [noise=true , address=45.91.116.6:4976 , piece_id=MI23FFRD5KRUMHI7WDNJMHQAE7KYHHZC5DWUKJXSS4FKLAD7JRGA, seg_pos={0 0}, upload=358970]: Done uploading piece. err: <nil> Dec 22 16:22:53.192627477 (408h53m53.063710522s) [noise=true , address=145.79.227.46:4601 , piece_id=52WOVLSR24N2W62HMKLSASXNSPJJBFZRDW6WGRZCAUC56KRI36RA, seg_pos={0 0}, upload=358970]: Done uploading piece. err: <nil> Dec 22 16:22:53.193088795 (408h53m53.06417183s) [noise=true , address=145.79.227.36:4519 , piece_id=RII4MLUE34LSXKEMLCTWG2PLEALFRSDBRRPMM2XWN33EQMQKRHLQ, seg_pos={0 0}, upload=358970]: Done uploading piece. err: <nil> Dec 22 16:22:53.195127166 (408h53m53.066210212s) [noise=true , address=145.79.227.17:4972 , piece_id=76V2OUI774LVGW7NSI6JIUIA457VLL5NFM77DW34QGHKI25SXF3Q, seg_pos={0 0}, upload=358970]: Done uploading piece. err: <nil> Dec 22 16:22:53.195590769 (408h53m53.066673804s) [noise=true , address=145.79.227.49:4339 , piece_id=CRXDJO422BRXC2DSXFAQCSSFLKBTV7UZLKVKXFPI7YPLYLJHFHHA, seg_pos={0 0}, upload=358970]: Done uploading piece. err: <nil> Dec 22 16:22:53.206057961 (408h53m53.077140996s) [noise=true , address=145.79.227.13:4803 , piece_id=COEHKUJEXAIDGTEZ2UKUUCO3PHKIQYYGUJKYL7C7BNMIAKJ2477A, seg_pos={0 0}, upload=358970]: Done uploading piece. err: <nil> Dec 22 16:22:53.219948020 (408h53m53.091031055s) [noise=true , address=145.79.227.52:4446 , piece_id=ABAYZHL5LDF2GTAS2XGQ7N3DCYOUQKQUC6PFHT2PRCXSKZIYASKA, seg_pos={0 0}, upload=358970]: Done uploading piece. err: <nil> Dec 22 16:22:53.229550604 (408h53m53.100633649s) [noise=true , address=145.79.227.9:4366 , piece_id=A5FEG74HZNCBYQK3EDA776ZJB4VTGV6ILQUQ6TBVNSPGX4LANSLA, seg_pos={0 0}, upload=358970]: Done uploading piece. err: <nil> Dec 22 16:22:53.318149072 (408h53m53.189232107s) [noise=true , address=145.79.227.45:4230 , piece_id=7WS62URQHMWTIJ2IEOZ4LZ3GCPAXEEPFFC62LKK6Z3RSWGUHHBEA, seg_pos={0 0}, upload=358970]: Done uploading piece. err: <nil> Dec 22 16:22:53.340811785 (408h53m53.211894821s) [noise=true , address=145.79.227.29:4942 , piece_id=EJNAXWNS7EEKOQ6EW3ZPILC5CMCXF5JCZFQCFBIR2SRYMO6LMCBQ, seg_pos={0 0}, upload=358970]: Done uploading piece. err: <nil> Dec 22 16:22:53.359006851 (408h53m53.230089896s) [noise=true , address=145.79.227.48:4303 , piece_id=43EJSUP74OKJFIKQIUZ3RIURVUZAA33GQW6HXCIB5S4BGC3PL5CQ, seg_pos={0 0}, upload=358970]: Done uploading piece. err: <nil> Dec 22 16:22:53.395653373 (408h53m53.266736418s) [noise=true , address=145.79.227.33:4869 , piece_id=KXXQEUEQIQFSAQ6ORHLFEOSH44FKL5OXAIEZYCKJG6WFGVE5E3LQ, seg_pos={0 0}, upload=358970]: Done uploading piece. err: <nil> Dec 22 16:22:53.395673721 (408h53m53.266756756s) [seg_pos={0 0}, upload=358970]: Segment reached optimal threshold of 13 pieces. Dec 22 16:22:53.395934689 (408h53m53.267017724s) [noise=true , address=145.79.227.30:4762 , piece_id=L6WJWMMF3G4VBFQ5BKZCDCGOJNEN4DKBDS3YSRQO76CKBITKLIVA, seg_pos={0 0}, upload=358970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1No1jHEK6YFSye92GGBri48Yv2DYUiESY6Gs3Z9PENjmBynS9b): protocol: expected piece hash; context canceled; EOF Dec 22 16:22:53.395944266 (408h53m53.267027302s) [noise=true , address=145.79.227.30:4762 , piece_id=L6WJWMMF3G4VBFQ5BKZCDCGOJNEN4DKBDS3YSRQO76CKBITKLIVA, seg_pos={0 0}, upload=358970]: Long tail cancellation due to optimal threshold Dec 22 16:22:53.396145630 (408h53m53.267228675s) [noise=true , address=45.91.116.12:4225 , piece_id=NB5OOJH5IW7SYGXUGJ2ELTS5NU5G7HXM6CREXCVHWU7TROQKK2LA, seg_pos={0 0}, upload=358970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: expected piece hash; context canceled; EOF Dec 22 16:22:53.396170778 (408h53m53.267253823s) [noise=true , address=45.91.116.12:4225 , piece_id=NB5OOJH5IW7SYGXUGJ2ELTS5NU5G7HXM6CREXCVHWU7TROQKK2LA, seg_pos={0 0}, upload=358970]: Long tail cancellation due to optimal threshold Dec 22 16:22:53.396182500 (408h53m53.267265535s) [noise=true , address=45.91.116.14:4189 , piece_id=43AR2356XPABF7ZR2547YV2A45S3WNQKFLLSXBAKWAASJXRHWUAQ, seg_pos={0 0}, upload=358970]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N): protocol: expected piece hash; context canceled; EOF Dec 22 16:22:53.396190135 (408h53m53.26727317s) [noise=true , address=45.91.116.14:4189 , piece_id=43AR2356XPABF7ZR2547YV2A45S3WNQKFLLSXBAKWAASJXRHWUAQ, seg_pos={0 0}, upload=358970]: Long tail cancellation due to optimal threshold Dec 22 16:22:53.396235522 (408h53m53.267318557s) [seg_pos={0 0}, upload=358970]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 22 16:22:53.396305765 (408h53m53.267388811s) [upload=358970]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 16:22:53.396310825 (408h53m53.26739386s) [upload=358970]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 16:22:53.697730199 (408h53m53.568813244s) [upload=358970]: Done uploading segments... Dec 22 16:22:53.697749516 (408h53m53.568832551s) [upload=358970]: Upload finished. err: <nil> Duration: 30.002444097s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........X............................X....................] duration: 5.219210016s trace: https://jaeger.dev.storj.io/trace/72726d8466c916ea FAIL (2025-12-22 16:23:02 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 5.219234003s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [XX..X...X..XXXX....X..X...XX.....XXXX......X.X.X.X...X....XX] duration: 30.003144108s trace: https://jaeger.dev.storj.io/trace/6964488f798ad07c FAIL (2025-12-22 16:22:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 6:22:19.902556842 (408h53m19.773639877s) [upload=358962]: Starting upload Dec 22 16:22:19.902625934 (408h53m19.773708969s) [upload=358962]: Done starting upload Dec 22 16:22:19.902839711 (408h53m19.773922756s) [upload=358962]: Uploading segments... Dec 22 16:22:19.902876963 (408h53m19.773960008s) [upload=358962]: Got next segment. Inline: false Dec 22 16:22:19.902916498 (408h53m19.773999533s) [upload=358962]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 16:22:19.902921227 (408h53m19.774004262s) [upload=358962]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 16:22:20.151439669 (408h53m20.022522704s) [seg_pos={0 0}, upload=358962]: Begin upload segment... Dec 22 16:22:20.151523890 (408h53m20.022606935s) [seg_pos={0 0}, upload=358962]: Done begin upload segment. Dec 22 16:22:20.151561972 (408h53m20.022645017s) [noise=true , address=145.79.227.52:4446 , piece_id=SCUCJCK7W3NGAW7QSSLCUV56CXRSLAIJEUGH45T43CKLGHLJNEQQ, seg_pos={0 0}, upload=358962]: Uploading piece... Dec 22 16:22:20.151596748 (408h53m20.022679793s) [noise=true , address=145.79.227.11:4729 , piece_id=6UVBRGFCDW45HU2YLZFLB7BTWIOLSE2PQUX4UNZKVTWOALUPVX7A, seg_pos={0 0}, upload=358962]: Uploading piece... Dec 22 16:22:20.152244532 (408h53m20.023327577s) [noise=true , address=145.79.227.47:4265 , piece_id=PVMNCBICIYODQVR7UMRNL2RGIBOJNLDSDGOE5P4YGD4A7P45XZOA, seg_pos={0 0}, upload=358962]: Uploading piece... Dec 22 16:22:20.152455223 (408h53m20.023538268s) [noise=true , address=45.91.116.10:4081 , piece_id=56PFHFPOHEDVP7MF2RIHODX6RF7TWTUYL7V2K2A3MD7FELNIIO5Q, seg_pos={0 0}, upload=358962]: Uploading piece... Dec 22 16:22:20.152557648 (408h53m20.023640683s) [noise=true , address=45.91.116.13:4153 , piece_id=YHE7GOWWUYLU3CMHFINDRZ6JMRL7GN5KSNMUQJZWIXNANS4S7U2Q, seg_pos={0 0}, upload=358962]: Uploading piece... Dec 22 16:22:20.152902404 (408h53m20.023985449s) [noise=true , address=45.91.116.4:4939 , piece_id=NJUL7HKWFO6UJGAIG5F4TU5JBRWEROX5RJUPH25UM25Q5OTAQMAA, seg_pos={0 0}, upload=358962]: Uploading piece... Dec 22 16:22:20.152964482 (408h53m20.024047527s) [noise=true , address=145.79.227.37:4566 , piece_id=NXMQTKAF2NISFANQOXAW6BWUZBBETJIY4SDJ5EUXPXCAE5FBGALA, seg_pos={0 0}, upload=358962]: Uploading piece... Dec 22 16:22:20.153167289 (408h53m20.024250324s) [noise=true , address=145.79.227.50:4374 , piece_id=XB6HKY5KHZTEFQDTN3QG6XONRWZ27CXOJGP4HJCKKZ5ZQWIITX3Q, seg_pos={0 0}, upload=358962]: Uploading piece... Dec 22 16:22:20.153421613 (408h53m20.024504658s) [noise=true , address=145.79.227.45:4231 , piece_id=HQR7WOOWPLRHNKUYMCJ36YK7KTZWT4RLTC3GW3XFQMA6SR4KOENQ, seg_pos={0 0}, upload=358962]: Uploading piece... Dec 22 16:22:20.153435289 (408h53m20.024518334s) [noise=true , address=77.90.31.11:4553 , piece_id=3IJICSSHB7PVHL4K737AC64BFJM3JBAQACNMYLLTPVP6XQ7PE33Q, seg_pos={0 0}, upload=358962]: Uploading piece... Dec 22 16:22:20.154161352 (408h53m20.025244387s) [noise=true , address=45.91.116.5:4299 , piece_id=FYZSQQA6T4CPNFEC76NAK2MPG2JFJFOTN3KGENMRKV2JTTDXD6JA, seg_pos={0 0}, upload=358962]: Uploading piece... Dec 22 16:22:20.154452506 (408h53m20.025535551s) [noise=true , address=145.79.227.35:4483 , piece_id=BVVHTQQ4FGNVTY3JCL2WPXDEEIMG4JVLJOPMLUNHPUCVWNUUCBRA, seg_pos={0 0}, upload=358962]: Uploading piece... Dec 22 16:22:20.155403437 (408h53m20.026486482s) [noise=true , address=145.79.227.41:4085 , piece_id=SETJHPUFO4CTNPWFHKGYB3UYYAUKJ4F3LXETPKEJLJKJJ66IV35A, seg_pos={0 0}, upload=358962]: Uploading piece... Dec 22 16:22:20.155719719 (408h53m20.026802754s) [noise=true , address=145.79.227.15:4876 , piece_id=W4AC74LJP65YR5LII3SA5XMFNRPCF2FS64AHXYZTYGFJQDXJEVVQ, seg_pos={0 0}, upload=358962]: Uploading piece... Dec 22 16:22:20.156154136 (408h53m20.027237182s) [noise=true , address=145.79.227.29:4941 , piece_id=C4DQXWVCFRQ23BUU2FI7HKMTMRCAVDEUHGUYL55TDX6YNFP4II7Q, seg_pos={0 0}, upload=358962]: Uploading piece... Dec 22 16:22:20.156346642 (408h53m20.027429678s) [noise=true , address=145.79.227.6:4258 , piece_id=QDENA2YNVNCWSP7PR4PRTBG4U2OKW74TZQF7UX67OBIVJUFN7K6A, seg_pos={0 0}, upload=358962]: Uploading piece... Dec 22 16:22:22.027597759 (408h53m21.898680804s) [upload=358962]: Next returned nil segment Dec 22 16:22:22.027619560 (408h53m21.898702605s) [upload=358962]: Waiting for error group managing segments... Dec 22 16:22:22.421397922 (408h53m22.292480957s) [noise=true , address=45.91.116.5:4299 , piece_id=FYZSQQA6T4CPNFEC76NAK2MPG2JFJFOTN3KGENMRKV2JTTDXD6JA, seg_pos={0 0}, upload=358962]: Done uploading piece. err: <nil> Dec 22 16:22:22.429781065 (408h53m22.3008641s) [noise=true , address=45.91.116.10:4081 , piece_id=56PFHFPOHEDVP7MF2RIHODX6RF7TWTUYL7V2K2A3MD7FELNIIO5Q, seg_pos={0 0}, upload=358962]: Done uploading piece. err: <nil> Dec 22 16:22:22.709032878 (408h53m22.580115923s) [noise=true , address=45.91.116.13:4153 , piece_id=YHE7GOWWUYLU3CMHFINDRZ6JMRL7GN5KSNMUQJZWIXNANS4S7U2Q, seg_pos={0 0}, upload=358962]: Done uploading piece. err: <nil> Dec 22 16:22:22.854856051 (408h53m22.725939086s) [noise=true , address=145.79.227.35:4483 , piece_id=BVVHTQQ4FGNVTY3JCL2WPXDEEIMG4JVLJOPMLUNHPUCVWNUUCBRA, seg_pos={0 0}, upload=358962]: Done uploading piece. err: <nil> Dec 22 16:22:22.859263153 (408h53m22.730346188s) [noise=true , address=145.79.227.50:4374 , piece_id=XB6HKY5KHZTEFQDTN3QG6XONRWZ27CXOJGP4HJCKKZ5ZQWIITX3Q, seg_pos={0 0}, upload=358962]: Done uploading piece. err: <nil> Dec 22 16:22:22.862229301 (408h53m22.733312366s) [noise=true , address=145.79.227.45:4231 , piece_id=HQR7WOOWPLRHNKUYMCJ36YK7KTZWT4RLTC3GW3XFQMA6SR4KOENQ, seg_pos={0 0}, upload=358962]: Done uploading piece. err: <nil> Dec 22 16:22:23.033093376 (408h53m22.904176421s) [noise=true , address=145.79.227.29:4941 , piece_id=C4DQXWVCFRQ23BUU2FI7HKMTMRCAVDEUHGUYL55TDX6YNFP4II7Q, seg_pos={0 0}, upload=358962]: Done uploading piece. err: <nil> Dec 22 16:22:23.087314211 (408h53m22.958397247s) [noise=true , address=145.79.227.47:4265 , piece_id=PVMNCBICIYODQVR7UMRNL2RGIBOJNLDSDGOE5P4YGD4A7P45XZOA, seg_pos={0 0}, upload=358962]: Done uploading piece. err: <nil> Dec 22 16:22:23.092887093 (408h53m22.963970128s) [noise=true , address=145.79.227.11:4729 , piece_id=6UVBRGFCDW45HU2YLZFLB7BTWIOLSE2PQUX4UNZKVTWOALUPVX7A, seg_pos={0 0}, upload=358962]: Done uploading piece. err: <nil> Dec 22 16:22:23.110047089 (408h53m22.981130134s) [noise=true , address=145.79.227.41:4085 , piece_id=SETJHPUFO4CTNPWFHKGYB3UYYAUKJ4F3LXETPKEJLJKJJ66IV35A, seg_pos={0 0}, upload=358962]: Done uploading piece. err: <nil> Dec 22 16:22:23.142023327 (408h53m23.013106363s) [noise=true , address=145.79.227.37:4566 , piece_id=NXMQTKAF2NISFANQOXAW6BWUZBBETJIY4SDJ5EUXPXCAE5FBGALA, seg_pos={0 0}, upload=358962]: Done uploading piece. err: <nil> Dec 22 16:22:23.181179457 (408h53m23.052262492s) [noise=true , address=145.79.227.15:4876 , piece_id=W4AC74LJP65YR5LII3SA5XMFNRPCF2FS64AHXYZTYGFJQDXJEVVQ, seg_pos={0 0}, upload=358962]: Done uploading piece. err: <nil> Dec 22 16:22:23.911036942 (408h53m23.782119987s) [noise=true , address=45.91.116.4:4939 , piece_id=NJUL7HKWFO6UJGAIG5F4TU5JBRWEROX5RJUPH25UM25Q5OTAQMAA, seg_pos={0 0}, upload=358962]: Done uploading piece. err: <nil> Dec 22 16:22:23.911059996 (408h53m23.782143041s) [seg_pos={0 0}, upload=358962]: Segment reached optimal threshold of 13 pieces. Dec 22 16:22:23.911310513 (408h53m23.782393548s) [noise=true , address=77.90.31.11:4553 , piece_id=3IJICSSHB7PVHL4K737AC64BFJM3JBAQACNMYLLTPVP6XQ7PE33Q, seg_pos={0 0}, upload=358962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Dec 22 16:22:23.911317927 (408h53m23.782400962s) [noise=true , address=77.90.31.11:4553 , piece_id=3IJICSSHB7PVHL4K737AC64BFJM3JBAQACNMYLLTPVP6XQ7PE33Q, seg_pos={0 0}, upload=358962]: Long tail cancellation due to optimal threshold Dec 22 16:22:23.911614873 (408h53m23.782697918s) [noise=true , address=145.79.227.52:4446 , piece_id=SCUCJCK7W3NGAW7QSSLCUV56CXRSLAIJEUGH45T43CKLGHLJNEQQ, seg_pos={0 0}, upload=358962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7): protocol: expected piece hash; context canceled; EOF Dec 22 16:22:23.911621986 (408h53m23.782705031s) [noise=true , address=145.79.227.52:4446 , piece_id=SCUCJCK7W3NGAW7QSSLCUV56CXRSLAIJEUGH45T43CKLGHLJNEQQ, seg_pos={0 0}, upload=358962]: Long tail cancellation due to optimal threshold Dec 22 16:22:23.911772062 (408h53m23.782855107s) [noise=true , address=145.79.227.6:4258 , piece_id=QDENA2YNVNCWSP7PR4PRTBG4U2OKW74TZQF7UX67OBIVJUFN7K6A, seg_pos={0 0}, upload=358962]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: write tcp 188.166.225.170:32822->145.79.227.6:4258: use of closed network connection; write tcp 188.166.225.170:32822->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32822->145.79.227.6:4258: use of closed network connection Dec 22 16:22:23.911779306 (408h53m23.782862341s) [noise=true , address=145.79.227.6:4258 , piece_id=QDENA2YNVNCWSP7PR4PRTBG4U2OKW74TZQF7UX67OBIVJUFN7K6A, seg_pos={0 0}, upload=358962]: Long tail cancellation due to optimal threshold Dec 22 16:22:23.911802490 (408h53m23.782885525s) [seg_pos={0 0}, upload=358962]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 22 16:22:23.911878584 (408h53m23.782961619s) [upload=358962]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 16:22:23.911884406 (408h53m23.782967461s) [upload=358962]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 16:22:24.199337566 (408h53m24.070420621s) [upload=358962]: Done uploading segments... Dec 22 16:22:24.199361973 (408h53m24.070445018s) [upload=358962]: Upload finished. err: <nil> Duration: 30.003157944s core uplink us-select-1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 475.542857ms target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/704b2d4a066e7708 FAIL (2025-12-22 16:22:40 UTC) orbiter warning: Get "https://eu1.storj.io/login": dial tcp 35.207.121.91:443: i/o timeout Duration: 475.573325ms core satellite eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 1.411367155s trace: https://jaeger.dev.storj.io/trace/6ea3fd74aff47917 FAIL (2025-12-22 16:22:36 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.41139588s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........X............................X...................] duration: 5.229317754s trace: https://jaeger.dev.storj.io/trace/72726d8466c916ea FAIL (2025-12-22 16:22:30 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 5.229372086s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [XXX..X...X..XXXX....X..X...XX.....XXXX......X.X.X.X...X....X] duration: 30.002399206s trace: https://jaeger.dev.storj.io/trace/22545d0d2e13741c FAIL (2025-12-22 16:22:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 22 16:21:49.653159500 (408h52m49.524242545s) [upload=358956]: Starting upload Dec 22 16:21:49.653255483 (408h52m49.524338528s) [upload=358956]: Done starting upload Dec 22 16:21:49.653432831 (408h52m49.524515876s) [upload=358956]: Uploading segments... Dec 22 16:21:49.653522051 (408h52m49.524605096s) [upload=358956]: Got next segment. Inline: false Dec 22 16:21:49.653562077 (408h52m49.524645122s) [upload=358956]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 16:21:49.653567006 (408h52m49.524650042s) [upload=358956]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 16:21:49.900658451 (408h52m49.771741496s) [seg_pos={0 0}, upload=358956]: Begin upload segment... Dec 22 16:21:49.900712934 (408h52m49.771795969s) [seg_pos={0 0}, upload=358956]: Done begin upload segment. Dec 22 16:21:49.900747500 (408h52m49.771830545s) [noise=true , address=145.79.227.37:4565 , piece_id=XPBHXMPYIOLF2Q2ENTQBXQM4A5UTTADKO7LY3S2SGFZTDOO2L43A, seg_pos={0 0}, upload=358956]: Uploading piece... Dec 22 16:21:49.900791555 (408h52m49.7718746s) [noise=true , address=45.91.116.12:4225 , piece_id=WISDEUV6ZMU2F3QPBZPV4ASU7MHKXYI7J4JC7O5JCGCFRMNTOEIA, seg_pos={0 0}, upload=358956]: Uploading piece... Dec 22 16:21:49.901359666 (408h52m49.772442711s) [noise=true , address=145.79.227.13:4803 , piece_id=IEXEKA4NTTIBIIAKLHVOPEVU5B5SDP7OSGXWWUUBTUUPKXA4CQXQ, seg_pos={0 0}, upload=358956]: Uploading piece... Dec 22 16:21:49.901367411 (408h52m49.772450446s) [noise=true , address=45.91.116.13:4155 , piece_id=4RKLO4TEHDICMQ2FZRIUF2ZCQH3OTZAYWK3I25KDWC7INSLUSNPQ, seg_pos={0 0}, upload=358956]: Uploading piece... Dec 22 16:21:49.901429299 (408h52m49.772512344s) [noise=true , address=45.91.116.7:4011 , piece_id=TFVM5G6ETVQRC6LV6VH2JIX5FCAG5CRERERKISSMPNX55URSS73A, seg_pos={0 0}, upload=358956]: Uploading piece... Dec 22 16:21:49.901789514 (408h52m49.772872559s) [noise=true , address=45.91.116.5:4300 , piece_id=PEA5D2CKS7UEN7QCZVREQ5FZPB65LDVUKKGZFN7X54IWPH7DBDIQ, seg_pos={0 0}, upload=358956]: Uploading piece... Dec 22 16:21:49.901859778 (408h52m49.772942813s) [noise=true , address=145.79.227.51:4411 , piece_id=LG5Q4QX2BSGHQSFDNGLT2S7MH25WLCZV3OTBQDIGCVQXX5KXUWTQ, seg_pos={0 0}, upload=358956]: Uploading piece... Dec 22 16:21:49.902232688 (408h52m49.773315734s) [noise=true , address=145.79.227.45:4229 , piece_id=LBEXS5ZCPJVDW4KNJKQB74KGSHNJ2MKLUDEQIOPYFNS27632MXTQ, seg_pos={0 0}, upload=358956]: Uploading piece... Dec 22 16:21:49.902274007 (408h52m49.773357042s) [noise=true , address=145.79.227.14:4837 , piece_id=AJVXHL7MEL27JPF24ISG5F7PJ3Q2SYC7JRMTXLGYUA2N3ULH7YWQ, seg_pos={0 0}, upload=358956]: Uploading piece... Dec 22 16:21:49.902632801 (408h52m49.773715846s) [noise=true , address=45.91.116.6:4975 , piece_id=AGVN2DQDS6Q54CEVN5KL5LM2BEVQ64J7XJ32OVFLQZ662C4QK3YQ, seg_pos={0 0}, upload=358956]: Uploading piece... Dec 22 16:21:49.902655073 (408h52m49.773738118s) [noise=true , address=145.79.227.15:4875 , piece_id=D3Y66DQNHTH7LO6XEBQ5NYF7BW6735Q7AUQCYQ45ZAQFIFPMKYTQ, seg_pos={0 0}, upload=358956]: Uploading piece... Dec 22 16:21:49.903043172 (408h52m49.774126207s) [noise=true , address=145.79.227.42:4123 , piece_id=BOUSNZT6A3BVLTVEGYM6MOWK6LDDJ7KSXSZ47RPYNJRUR2G44HJA, seg_pos={0 0}, upload=358956]: Uploading piece... Dec 22 16:21:49.903055215 (408h52m49.77413825s) [noise=true , address=145.79.227.17:4972 , piece_id=DY4Y5XCZQB7DKAOUIZYWND6HPGKDW2WLJGDLFC54577B62WDJM5Q, seg_pos={0 0}, upload=358956]: Uploading piece... Dec 22 16:21:49.903443394 (408h52m49.774526429s) [noise=true , address=145.79.227.41:4086 , piece_id=4TGMGYA7TLZ4U3NQ5JXMHHHG2ZECFPGISFZGWWLK65BBZJB4EKNA, seg_pos={0 0}, upload=358956]: Uploading piece... Dec 22 16:21:49.904762135 (408h52m49.77584518s) [noise=true , address=45.91.116.14:4190 , piece_id=FGDHQYXKUZ6ROQNK66CJFRM3QV5RFKGIY7LFIHLKOXXF5GKLP5TQ, seg_pos={0 0}, upload=358956]: Uploading piece... Dec 22 16:21:49.910431160 (408h52m49.781514245s) [noise=true , address=145.79.227.35:4483 , piece_id=KCV3Y4CYWZQOGALSRYWOBWKMR2HZAQYKQZZRVJALO5VGIHBIBK7Q, seg_pos={0 0}, upload=358956]: Uploading piece... Dec 22 16:21:51.809208040 (408h52m51.680291085s) [upload=358956]: Next returned nil segment Dec 22 16:21:51.809220293 (408h52m51.680303328s) [upload=358956]: Waiting for error group managing segments... Dec 22 16:21:52.130767344 (408h52m52.001850379s) [noise=true , address=45.91.116.6:4975 , piece_id=AGVN2DQDS6Q54CEVN5KL5LM2BEVQ64J7XJ32OVFLQZ662C4QK3YQ, seg_pos={0 0}, upload=358956]: Done uploading piece. err: <nil> Dec 22 16:21:52.156669776 (408h52m52.027752811s) [noise=true , address=45.91.116.5:4300 , piece_id=PEA5D2CKS7UEN7QCZVREQ5FZPB65LDVUKKGZFN7X54IWPH7DBDIQ, seg_pos={0 0}, upload=358956]: Done uploading piece. err: <nil> Dec 22 16:21:52.176195958 (408h52m52.047278993s) [noise=true , address=45.91.116.12:4225 , piece_id=WISDEUV6ZMU2F3QPBZPV4ASU7MHKXYI7J4JC7O5JCGCFRMNTOEIA, seg_pos={0 0}, upload=358956]: Done uploading piece. err: <nil> Dec 22 16:21:52.179687998 (408h52m52.050771043s) [noise=true , address=45.91.116.13:4155 , piece_id=4RKLO4TEHDICMQ2FZRIUF2ZCQH3OTZAYWK3I25KDWC7INSLUSNPQ, seg_pos={0 0}, upload=358956]: Done uploading piece. err: <nil> Dec 22 16:21:52.208105319 (408h52m52.079188354s) [noise=true , address=45.91.116.7:4011 , piece_id=TFVM5G6ETVQRC6LV6VH2JIX5FCAG5CRERERKISSMPNX55URSS73A, seg_pos={0 0}, upload=358956]: Done uploading piece. err: <nil> Dec 22 16:21:52.758584026 (408h52m52.629667071s) [noise=true , address=145.79.227.41:4086 , piece_id=4TGMGYA7TLZ4U3NQ5JXMHHHG2ZECFPGISFZGWWLK65BBZJB4EKNA, seg_pos={0 0}, upload=358956]: Done uploading piece. err: <nil> Dec 22 16:21:52.761207271 (408h52m52.632290316s) [noise=true , address=145.79.227.51:4411 , piece_id=LG5Q4QX2BSGHQSFDNGLT2S7MH25WLCZV3OTBQDIGCVQXX5KXUWTQ, seg_pos={0 0}, upload=358956]: Done uploading piece. err: <nil> Dec 22 16:21:52.776160964 (408h52m52.647244009s) [noise=true , address=145.79.227.14:4837 , piece_id=AJVXHL7MEL27JPF24ISG5F7PJ3Q2SYC7JRMTXLGYUA2N3ULH7YWQ, seg_pos={0 0}, upload=358956]: Done uploading piece. err: <nil> Dec 22 16:21:52.777326915 (408h52m52.64840996s) [noise=true , address=145.79.227.42:4123 , piece_id=BOUSNZT6A3BVLTVEGYM6MOWK6LDDJ7KSXSZ47RPYNJRUR2G44HJA, seg_pos={0 0}, upload=358956]: Done uploading piece. err: <nil> Dec 22 16:21:52.778968972 (408h52m52.650052017s) [noise=true , address=145.79.227.15:4875 , piece_id=D3Y66DQNHTH7LO6XEBQ5NYF7BW6735Q7AUQCYQ45ZAQFIFPMKYTQ, seg_pos={0 0}, upload=358956]: Done uploading piece. err: <nil> Dec 22 16:21:52.783106019 (408h52m52.654189055s) [noise=true , address=145.79.227.35:4483 , piece_id=KCV3Y4CYWZQOGALSRYWOBWKMR2HZAQYKQZZRVJALO5VGIHBIBK7Q, seg_pos={0 0}, upload=358956]: Done uploading piece. err: <nil> Dec 22 16:21:52.796466029 (408h52m52.667549075s) [noise=true , address=145.79.227.37:4565 , piece_id=XPBHXMPYIOLF2Q2ENTQBXQM4A5UTTADKO7LY3S2SGFZTDOO2L43A, seg_pos={0 0}, upload=358956]: Done uploading piece. err: <nil> Dec 22 16:21:52.894127273 (408h52m52.765210318s) [noise=true , address=45.91.116.14:4190 , piece_id=FGDHQYXKUZ6ROQNK66CJFRM3QV5RFKGIY7LFIHLKOXXF5GKLP5TQ, seg_pos={0 0}, upload=358956]: Done uploading piece. err: <nil> Dec 22 16:21:52.894156769 (408h52m52.765239834s) [seg_pos={0 0}, upload=358956]: Segment reached optimal threshold of 13 pieces. Dec 22 16:21:52.894453094 (408h52m52.765536139s) [noise=true , address=145.79.227.13:4803 , piece_id=IEXEKA4NTTIBIIAKLHVOPEVU5B5SDP7OSGXWWUUBTUUPKXA4CQXQ, seg_pos={0 0}, upload=358956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: expected piece hash; context canceled; EOF Dec 22 16:21:52.894465537 (408h52m52.765548582s) [noise=true , address=145.79.227.13:4803 , piece_id=IEXEKA4NTTIBIIAKLHVOPEVU5B5SDP7OSGXWWUUBTUUPKXA4CQXQ, seg_pos={0 0}, upload=358956]: Long tail cancellation due to optimal threshold Dec 22 16:21:52.894835612 (408h52m52.765918647s) [noise=true , address=145.79.227.17:4972 , piece_id=DY4Y5XCZQB7DKAOUIZYWND6HPGKDW2WLJGDLFC54577B62WDJM5Q, seg_pos={0 0}, upload=358956]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX): protocol: expected piece hash; context canceled; EOF Dec 22 16:21:52.894845260 (408h52m52.765928305s) [noise=true , address=145.79.227.17:4972 , piece_id=DY4Y5XCZQB7DKAOUIZYWND6HPGKDW2WLJGDLFC54577B62WDJM5Q, seg_pos={0 0}, upload=358956]: Long tail cancellation due to optimal threshold Dec 22 16:21:52.894960820 (408h52m52.766043855s) [noise=true , address=145.79.227.45:4229 , piece_id=LBEXS5ZCPJVDW4KNJKQB74KGSHNJ2MKLUDEQIOPYFNS27632MXTQ, seg_pos={0 0}, upload=358956]: Done uploading piece. err: <nil> Dec 22 16:21:52.895016366 (408h52m52.766099411s) [seg_pos={0 0}, upload=358956]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Dec 22 16:21:52.895107440 (408h52m52.766190485s) [upload=358956]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 16:21:52.895112670 (408h52m52.766195715s) [upload=358956]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 16:21:53.189737225 (408h52m53.06082026s) [upload=358956]: Done uploading segments... Dec 22 16:21:53.189760149 (408h52m53.060843194s) [upload=358956]: Upload finished. err: <nil> Duration: 30.002424765s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 1.380589531s trace: https://jaeger.dev.storj.io/trace/6ea3fd74aff47917 FAIL (2025-12-22 16:21:48 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.380614328s core uplink eu1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 469.71564ms target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/704b2d4a066e7708 FAIL (2025-12-22 16:21:44 UTC) orbiter warning: Get "https://eu1.storj.io/login": dial tcp 35.207.121.91:443: i/o timeout Duration: 469.790703ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [.XXX..X...X..XXXX....X..X...XX.....XXXX......X.X.X.X...X....] duration: 29.393682975s trace: https://jaeger.dev.storj.io/trace/6eb5416200619b22 FAIL (2025-12-22 16:21:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 22 16:18:09.037402914 (408h49m8.908485959s) [upload=358898]: Starting upload Dec 22 16:18:09.037510439 (408h49m8.908593504s) [upload=358898]: Done starting upload Dec 22 16:18:09.037545956 (408h49m8.908629002s) [upload=358898]: Uploading segments... Dec 22 16:18:09.037759613 (408h49m8.908842658s) [upload=358898]: Got next segment. Inline: false Dec 22 16:18:09.037807204 (408h49m8.90889025s) [upload=358898]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 16:18:09.037813126 (408h49m8.908896171s) [upload=358898]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 16:18:09.289696365 (408h49m9.1607794s) [seg_pos={0 0}, upload=358898]: Begin upload segment... Dec 22 16:18:09.289741130 (408h49m9.160824175s) [seg_pos={0 0}, upload=358898]: Done begin upload segment. Dec 22 16:18:09.289767701 (408h49m9.160850746s) [noise=true , address=145.79.227.13:4803 , piece_id=NS57HINS2HYB56GFXJWNFCVB6BOA7AA5MTHOW2YVLFK3PTPDF5HQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.289986888 (408h49m9.161069923s) [noise=true , address=77.90.31.14:4662 , piece_id=CQSKEBWWSNDMYXCVCYE6HKVEJ4BXSM54ABTDTLLDFPSBZB34BI7Q, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.290568395 (408h49m9.16165144s) [noise=true , address=145.79.227.51:4409 , piece_id=GFDYLOEWMOIO2MSAZDXIOBXR6P2DSEAX56CDF4T7SLGPXS3AO6BA, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.291017320 (408h49m9.162100355s) [noise=true , address=45.91.116.14:4191 , piece_id=TYNCODGLLAGFMNAGPN6X7HJKRAZ6IEAFIUQIS3ZYOFMUTYWUQJGA, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.291467107 (408h49m9.162550152s) [noise=true , address=77.90.31.19:4844 , piece_id=P3ARVIRV6DHLSXYK2QBBSDPRCEAYLSHJLEAGIAJ6UYLBGSQRBK6A, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.292000592 (408h49m9.163083637s) [noise=true , address=145.79.227.48:4301 , piece_id=DAM6FOLQWAC2ZHA274NPGL743KI6QQUNPGQ5ETEPTFV2UCS2C4WA, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.292520292 (408h49m9.163603337s) [noise=true , address=145.79.227.42:4121 , piece_id=CRAOIQLWBTENWZN6AMDWSMKMXEAB36ZHELNL7737HALMZSPH3DSQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.293926140 (408h49m9.165009185s) [noise=true , address=145.79.227.41:4086 , piece_id=CIW56LRG4OSSKY7HDHKC352FN77KXADPWFJQ6AH44QWGRKCS2Y5Q, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.294123896 (408h49m9.165206931s) [noise=true , address=145.79.227.46:4602 , piece_id=SN22WFK7KRBZXRGQPVLRZ4CVRPU6Q34XIAX3Y4BJ4HSH3ZRWO5XQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.295190838 (408h49m9.166273873s) [noise=true , address=145.79.227.35:4482 , piece_id=PWJDZXJ2BWCNLDXIEHNOQHTDZ527TIBXIZQWDV7QR4G6K5PM5GFQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.295672685 (408h49m9.16675573s) [noise=true , address=145.79.227.8:4329 , piece_id=PIMLYVB45QTX3RKSSF5HVO7DVWSTEDBPUPEI4YYDKL55OK3G7GFA, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.296512795 (408h49m9.16759583s) [noise=true , address=145.79.227.14:4837 , piece_id=UGUTYFGPK3WF3N3DMBWHQHBNWSU7DXHXSPVTUIITKCMY2M7PFR6A, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.297667263 (408h49m9.168750298s) [noise=true , address=145.79.227.47:4265 , piece_id=5R37XFDZNHGQYJ2FINHLAHDB4MXJWMAYNB2HYHO4B5UM4ETORBZQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.298831309 (408h49m9.169914344s) [noise=true , address=145.79.227.37:4566 , piece_id=C4XIYBXUXTGL6AQMNSIU2SCC6TZNV54NYQLAOYVEGHFBFOO7S3LQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.300152595 (408h49m9.17123564s) [noise=true , address=145.79.227.50:4375 , piece_id=UDPHQP42KHKH3CBIAILJM6BJ34UAA2D7GA3BIXDK5IOYBLWHO7YQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.301524037 (408h49m9.172607072s) [noise=true , address=145.79.227.32:4835 , piece_id=6XVWDEHT4KKJUQMXFFVJU7EGZTUXIHFGRXC2CI66ZA3ERBJXEMVA, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:11.170954938 (408h49m11.042037983s) [upload=358898]: Next returned nil segment Dec 22 16:18:11.170982821 (408h49m11.042065866s) [upload=358898]: Waiting for error group managing segments... Dec 22 16:18:11.470411986 (408h49m11.341495021s) [noise=true , address=77.90.31.14:4662 , piece_id=CQSKEBWWSNDMYXCVCYE6HKVEJ4BXSM54ABTDTLLDFPSBZB34BI7Q, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:11.514534372 (408h49m11.385617417s) [noise=true , address=45.91.116.14:4191 , piece_id=TYNCODGLLAGFMNAGPN6X7HJKRAZ6IEAFIUQIS3ZYOFMUTYWUQJGA, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:11.590140129 (408h49m11.461223174s) [noise=true , address=77.90.31.19:4844 , piece_id=P3ARVIRV6DHLSXYK2QBBSDPRCEAYLSHJLEAGIAJ6UYLBGSQRBK6A, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.029558086 (408h49m11.900641131s) [noise=true , address=145.79.227.48:4301 , piece_id=DAM6FOLQWAC2ZHA274NPGL743KI6QQUNPGQ5ETEPTFV2UCS2C4WA, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.050418598 (408h49m11.921501633s) [noise=true , address=145.79.227.8:4329 , piece_id=PIMLYVB45QTX3RKSSF5HVO7DVWSTEDBPUPEI4YYDKL55OK3G7GFA, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.063480270 (408h49m11.934563326s) [noise=true , address=145.79.227.47:4265 , piece_id=5R37XFDZNHGQYJ2FINHLAHDB4MXJWMAYNB2HYHO4B5UM4ETORBZQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.063617061 (408h49m11.934700106s) [noise=true , address=145.79.227.35:4482 , piece_id=PWJDZXJ2BWCNLDXIEHNOQHTDZ527TIBXIZQWDV7QR4G6K5PM5GFQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.075770333 (408h49m11.946853378s) [noise=true , address=145.79.227.46:4602 , piece_id=SN22WFK7KRBZXRGQPVLRZ4CVRPU6Q34XIAX3Y4BJ4HSH3ZRWO5XQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.114183588 (408h49m11.985266653s) [noise=true , address=145.79.227.37:4566 , piece_id=C4XIYBXUXTGL6AQMNSIU2SCC6TZNV54NYQLAOYVEGHFBFOO7S3LQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.128980844 (408h49m12.000063879s) [noise=true , address=145.79.227.50:4375 , piece_id=UDPHQP42KHKH3CBIAILJM6BJ34UAA2D7GA3BIXDK5IOYBLWHO7YQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.151198050 (408h49m12.022281096s) [noise=true , address=145.79.227.13:4803 , piece_id=NS57HINS2HYB56GFXJWNFCVB6BOA7AA5MTHOW2YVLFK3PTPDF5HQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.156165298 (408h49m12.027248343s) [noise=true , address=145.79.227.42:4121 , piece_id=CRAOIQLWBTENWZN6AMDWSMKMXEAB36ZHELNL7737HALMZSPH3DSQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.156486050 (408h49m12.027569095s) [noise=true , address=145.79.227.41:4086 , piece_id=CIW56LRG4OSSKY7HDHKC352FN77KXADPWFJQ6AH44QWGRKCS2Y5Q, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.156512820 (408h49m12.027595865s) [seg_pos={0 0}, upload=358898]: Segment reached optimal threshold of 13 pieces. Dec 22 16:18:12.156773717 (408h49m12.027856762s) [noise=true , address=145.79.227.14:4837 , piece_id=UGUTYFGPK3WF3N3DMBWHQHBNWSU7DXHXSPVTUIITKCMY2M7PFR6A, seg_pos={0 0}, upload=358898]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: expected piece hash; context canceled; EOF Dec 22 16:18:12.156786501 (408h49m12.027869546s) [noise=true , address=145.79.227.14:4837 , piece_id=UGUTYFGPK3WF3N3DMBWHQHBNWSU7DXHXSPVTUIITKCMY2M7PFR6A, seg_pos={0 0}, upload=358898]: Long tail cancellation due to optimal threshold Dec 22 16:18:12.156853008 (408h49m12.027936063s) [noise=true , address=145.79.227.32:4835 , piece_id=6XVWDEHT4KKJUQMXFFVJU7EGZTUXIHFGRXC2CI66ZA3ERBJXEMVA, seg_pos={0 0}, upload=358898]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y): protocol: expected piece hash; context canceled; EOF Dec 22 16:18:12.156864379 (408h49m12.027947434s) [noise=true , address=145.79.227.32:4835 , piece_id=6XVWDEHT4KKJUQMXFFVJU7EGZTUXIHFGRXC2CI66ZA3ERBJXEMVA, seg_pos={0 0}, upload=358898]: Long tail cancellation due to optimal threshold Dec 22 16:18:12.157100860 (408h49m12.028183905s) [noise=true , address=145.79.227.51:4409 , piece_id=GFDYLOEWMOIO2MSAZDXIOBXR6P2DSEAX56CDF4T7SLGPXS3AO6BA, seg_pos={0 0}, upload=358898]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: expected piece hash; context canceled; EOF Dec 22 16:18:12.157115689 (408h49m12.028198754s) [noise=true , address=145.79.227.51:4409 , piece_id=GFDYLOEWMOIO2MSAZDXIOBXR6P2DSEAX56CDF4T7SLGPXS3AO6BA, seg_pos={0 0}, upload=358898]: Long tail cancellation due to optimal threshold Dec 22 16:18:12.157144382 (408h49m12.028227418s) [seg_pos={0 0}, upload=358898]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 22 16:18:12.157297575 (408h49m12.02838063s) [upload=358898]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 16:18:12.157310810 (408h49m12.028393855s) [upload=358898]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 16:18:12.458231164 (408h49m12.32931423s) [upload=358898]: Done uploading segments... Dec 22 16:18:12.458268646 (408h49m12.329351691s) [upload=358898]: Upload finished. err: <nil> Duration: 29.393714605s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 1.445287727s trace: https://jaeger.dev.storj.io/trace/6ea3fd74aff47917 FAIL (2025-12-22 16:21:31 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.445318765s core uplink eu1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 472.326179ms target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/704b2d4a066e7708 FAIL (2025-12-22 16:21:22 UTC) orbiter warning: Get "https://eu1.storj.io/login": dial tcp 35.207.121.91:443: i/o timeout Duration: 472.343974ms core satellite eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............X............................X..................] duration: 5.434918475s trace: https://jaeger.dev.storj.io/trace/72726d8466c916ea FAIL (2025-12-22 16:21:20 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 5.434945997s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 1.381807029s trace: https://jaeger.dev.storj.io/trace/6ea3fd74aff47917 FAIL (2025-12-22 16:21:06 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.381836526s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [..XXX..X...X..XXXX....X..X...XX.....XXXX......X.X.X.X...X...] duration: 8.19465588s trace: https://jaeger.dev.storj.io/trace/6eb5416200619b22 FAIL (2025-12-22 16:20:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 22 16:18:09.037402914 (408h49m8.908485959s) [upload=358898]: Starting upload Dec 22 16:18:09.037510439 (408h49m8.908593504s) [upload=358898]: Done starting upload Dec 22 16:18:09.037545956 (408h49m8.908629002s) [upload=358898]: Uploading segments... Dec 22 16:18:09.037759613 (408h49m8.908842658s) [upload=358898]: Got next segment. Inline: false Dec 22 16:18:09.037807204 (408h49m8.90889025s) [upload=358898]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 16:18:09.037813126 (408h49m8.908896171s) [upload=358898]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 16:18:09.289696365 (408h49m9.1607794s) [seg_pos={0 0}, upload=358898]: Begin upload segment... Dec 22 16:18:09.289741130 (408h49m9.160824175s) [seg_pos={0 0}, upload=358898]: Done begin upload segment. Dec 22 16:18:09.289767701 (408h49m9.160850746s) [noise=true , address=145.79.227.13:4803 , piece_id=NS57HINS2HYB56GFXJWNFCVB6BOA7AA5MTHOW2YVLFK3PTPDF5HQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.289986888 (408h49m9.161069923s) [noise=true , address=77.90.31.14:4662 , piece_id=CQSKEBWWSNDMYXCVCYE6HKVEJ4BXSM54ABTDTLLDFPSBZB34BI7Q, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.290568395 (408h49m9.16165144s) [noise=true , address=145.79.227.51:4409 , piece_id=GFDYLOEWMOIO2MSAZDXIOBXR6P2DSEAX56CDF4T7SLGPXS3AO6BA, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.291017320 (408h49m9.162100355s) [noise=true , address=45.91.116.14:4191 , piece_id=TYNCODGLLAGFMNAGPN6X7HJKRAZ6IEAFIUQIS3ZYOFMUTYWUQJGA, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.291467107 (408h49m9.162550152s) [noise=true , address=77.90.31.19:4844 , piece_id=P3ARVIRV6DHLSXYK2QBBSDPRCEAYLSHJLEAGIAJ6UYLBGSQRBK6A, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.292000592 (408h49m9.163083637s) [noise=true , address=145.79.227.48:4301 , piece_id=DAM6FOLQWAC2ZHA274NPGL743KI6QQUNPGQ5ETEPTFV2UCS2C4WA, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.292520292 (408h49m9.163603337s) [noise=true , address=145.79.227.42:4121 , piece_id=CRAOIQLWBTENWZN6AMDWSMKMXEAB36ZHELNL7737HALMZSPH3DSQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.293926140 (408h49m9.165009185s) [noise=true , address=145.79.227.41:4086 , piece_id=CIW56LRG4OSSKY7HDHKC352FN77KXADPWFJQ6AH44QWGRKCS2Y5Q, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.294123896 (408h49m9.165206931s) [noise=true , address=145.79.227.46:4602 , piece_id=SN22WFK7KRBZXRGQPVLRZ4CVRPU6Q34XIAX3Y4BJ4HSH3ZRWO5XQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.295190838 (408h49m9.166273873s) [noise=true , address=145.79.227.35:4482 , piece_id=PWJDZXJ2BWCNLDXIEHNOQHTDZ527TIBXIZQWDV7QR4G6K5PM5GFQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.295672685 (408h49m9.16675573s) [noise=true , address=145.79.227.8:4329 , piece_id=PIMLYVB45QTX3RKSSF5HVO7DVWSTEDBPUPEI4YYDKL55OK3G7GFA, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.296512795 (408h49m9.16759583s) [noise=true , address=145.79.227.14:4837 , piece_id=UGUTYFGPK3WF3N3DMBWHQHBNWSU7DXHXSPVTUIITKCMY2M7PFR6A, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.297667263 (408h49m9.168750298s) [noise=true , address=145.79.227.47:4265 , piece_id=5R37XFDZNHGQYJ2FINHLAHDB4MXJWMAYNB2HYHO4B5UM4ETORBZQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.298831309 (408h49m9.169914344s) [noise=true , address=145.79.227.37:4566 , piece_id=C4XIYBXUXTGL6AQMNSIU2SCC6TZNV54NYQLAOYVEGHFBFOO7S3LQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.300152595 (408h49m9.17123564s) [noise=true , address=145.79.227.50:4375 , piece_id=UDPHQP42KHKH3CBIAILJM6BJ34UAA2D7GA3BIXDK5IOYBLWHO7YQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.301524037 (408h49m9.172607072s) [noise=true , address=145.79.227.32:4835 , piece_id=6XVWDEHT4KKJUQMXFFVJU7EGZTUXIHFGRXC2CI66ZA3ERBJXEMVA, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:11.170954938 (408h49m11.042037983s) [upload=358898]: Next returned nil segment Dec 22 16:18:11.170982821 (408h49m11.042065866s) [upload=358898]: Waiting for error group managing segments... Dec 22 16:18:11.470411986 (408h49m11.341495021s) [noise=true , address=77.90.31.14:4662 , piece_id=CQSKEBWWSNDMYXCVCYE6HKVEJ4BXSM54ABTDTLLDFPSBZB34BI7Q, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:11.514534372 (408h49m11.385617417s) [noise=true , address=45.91.116.14:4191 , piece_id=TYNCODGLLAGFMNAGPN6X7HJKRAZ6IEAFIUQIS3ZYOFMUTYWUQJGA, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:11.590140129 (408h49m11.461223174s) [noise=true , address=77.90.31.19:4844 , piece_id=P3ARVIRV6DHLSXYK2QBBSDPRCEAYLSHJLEAGIAJ6UYLBGSQRBK6A, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.029558086 (408h49m11.900641131s) [noise=true , address=145.79.227.48:4301 , piece_id=DAM6FOLQWAC2ZHA274NPGL743KI6QQUNPGQ5ETEPTFV2UCS2C4WA, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.050418598 (408h49m11.921501633s) [noise=true , address=145.79.227.8:4329 , piece_id=PIMLYVB45QTX3RKSSF5HVO7DVWSTEDBPUPEI4YYDKL55OK3G7GFA, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.063480270 (408h49m11.934563326s) [noise=true , address=145.79.227.47:4265 , piece_id=5R37XFDZNHGQYJ2FINHLAHDB4MXJWMAYNB2HYHO4B5UM4ETORBZQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.063617061 (408h49m11.934700106s) [noise=true , address=145.79.227.35:4482 , piece_id=PWJDZXJ2BWCNLDXIEHNOQHTDZ527TIBXIZQWDV7QR4G6K5PM5GFQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.075770333 (408h49m11.946853378s) [noise=true , address=145.79.227.46:4602 , piece_id=SN22WFK7KRBZXRGQPVLRZ4CVRPU6Q34XIAX3Y4BJ4HSH3ZRWO5XQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.114183588 (408h49m11.985266653s) [noise=true , address=145.79.227.37:4566 , piece_id=C4XIYBXUXTGL6AQMNSIU2SCC6TZNV54NYQLAOYVEGHFBFOO7S3LQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.128980844 (408h49m12.000063879s) [noise=true , address=145.79.227.50:4375 , piece_id=UDPHQP42KHKH3CBIAILJM6BJ34UAA2D7GA3BIXDK5IOYBLWHO7YQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.151198050 (408h49m12.022281096s) [noise=true , address=145.79.227.13:4803 , piece_id=NS57HINS2HYB56GFXJWNFCVB6BOA7AA5MTHOW2YVLFK3PTPDF5HQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.156165298 (408h49m12.027248343s) [noise=true , address=145.79.227.42:4121 , piece_id=CRAOIQLWBTENWZN6AMDWSMKMXEAB36ZHELNL7737HALMZSPH3DSQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.156486050 (408h49m12.027569095s) [noise=true , address=145.79.227.41:4086 , piece_id=CIW56LRG4OSSKY7HDHKC352FN77KXADPWFJQ6AH44QWGRKCS2Y5Q, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.156512820 (408h49m12.027595865s) [seg_pos={0 0}, upload=358898]: Segment reached optimal threshold of 13 pieces. Dec 22 16:18:12.156773717 (408h49m12.027856762s) [noise=true , address=145.79.227.14:4837 , piece_id=UGUTYFGPK3WF3N3DMBWHQHBNWSU7DXHXSPVTUIITKCMY2M7PFR6A, seg_pos={0 0}, upload=358898]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: expected piece hash; context canceled; EOF Dec 22 16:18:12.156786501 (408h49m12.027869546s) [noise=true , address=145.79.227.14:4837 , piece_id=UGUTYFGPK3WF3N3DMBWHQHBNWSU7DXHXSPVTUIITKCMY2M7PFR6A, seg_pos={0 0}, upload=358898]: Long tail cancellation due to optimal threshold Dec 22 16:18:12.156853008 (408h49m12.027936063s) [noise=true , address=145.79.227.32:4835 , piece_id=6XVWDEHT4KKJUQMXFFVJU7EGZTUXIHFGRXC2CI66ZA3ERBJXEMVA, seg_pos={0 0}, upload=358898]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y): protocol: expected piece hash; context canceled; EOF Dec 22 16:18:12.156864379 (408h49m12.027947434s) [noise=true , address=145.79.227.32:4835 , piece_id=6XVWDEHT4KKJUQMXFFVJU7EGZTUXIHFGRXC2CI66ZA3ERBJXEMVA, seg_pos={0 0}, upload=358898]: Long tail cancellation due to optimal threshold Dec 22 16:18:12.157100860 (408h49m12.028183905s) [noise=true , address=145.79.227.51:4409 , piece_id=GFDYLOEWMOIO2MSAZDXIOBXR6P2DSEAX56CDF4T7SLGPXS3AO6BA, seg_pos={0 0}, upload=358898]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: expected piece hash; context canceled; EOF Dec 22 16:18:12.157115689 (408h49m12.028198754s) [noise=true , address=145.79.227.51:4409 , piece_id=GFDYLOEWMOIO2MSAZDXIOBXR6P2DSEAX56CDF4T7SLGPXS3AO6BA, seg_pos={0 0}, upload=358898]: Long tail cancellation due to optimal threshold Dec 22 16:18:12.157144382 (408h49m12.028227418s) [seg_pos={0 0}, upload=358898]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 22 16:18:12.157297575 (408h49m12.02838063s) [upload=358898]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 16:18:12.157310810 (408h49m12.028393855s) [upload=358898]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 16:18:12.458231164 (408h49m12.32931423s) [upload=358898]: Done uploading segments... Dec 22 16:18:12.458268646 (408h49m12.329351691s) [upload=358898]: Upload finished. err: <nil> Duration: 8.194690667s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 1.379999457s trace: https://jaeger.dev.storj.io/trace/6ea3fd74aff47917 FAIL (2025-12-22 16:20:41 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.380023153s core uplink eu1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 468.049737ms target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/704b2d4a066e7708 FAIL (2025-12-22 16:20:34 UTC) orbiter warning: Get "https://eu1.storj.io/login": dial tcp 35.207.121.91:443: i/o timeout Duration: 468.078773ms core satellite eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............X............................X.................] duration: 5.798908383s trace: https://jaeger.dev.storj.io/trace/72726d8466c916ea FAIL (2025-12-22 16:20:15 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 5.798930585s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [...XXX..X...X..XXXX....X..X...XX.....XXXX......X.X.X.X...X..] duration: 8.740134496s trace: https://jaeger.dev.storj.io/trace/6eb5416200619b22 FAIL (2025-12-22 16:20:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 22 16:18:09.037402914 (408h49m8.908485959s) [upload=358898]: Starting upload Dec 22 16:18:09.037510439 (408h49m8.908593504s) [upload=358898]: Done starting upload Dec 22 16:18:09.037545956 (408h49m8.908629002s) [upload=358898]: Uploading segments... Dec 22 16:18:09.037759613 (408h49m8.908842658s) [upload=358898]: Got next segment. Inline: false Dec 22 16:18:09.037807204 (408h49m8.90889025s) [upload=358898]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 16:18:09.037813126 (408h49m8.908896171s) [upload=358898]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 16:18:09.289696365 (408h49m9.1607794s) [seg_pos={0 0}, upload=358898]: Begin upload segment... Dec 22 16:18:09.289741130 (408h49m9.160824175s) [seg_pos={0 0}, upload=358898]: Done begin upload segment. Dec 22 16:18:09.289767701 (408h49m9.160850746s) [noise=true , address=145.79.227.13:4803 , piece_id=NS57HINS2HYB56GFXJWNFCVB6BOA7AA5MTHOW2YVLFK3PTPDF5HQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.289986888 (408h49m9.161069923s) [noise=true , address=77.90.31.14:4662 , piece_id=CQSKEBWWSNDMYXCVCYE6HKVEJ4BXSM54ABTDTLLDFPSBZB34BI7Q, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.290568395 (408h49m9.16165144s) [noise=true , address=145.79.227.51:4409 , piece_id=GFDYLOEWMOIO2MSAZDXIOBXR6P2DSEAX56CDF4T7SLGPXS3AO6BA, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.291017320 (408h49m9.162100355s) [noise=true , address=45.91.116.14:4191 , piece_id=TYNCODGLLAGFMNAGPN6X7HJKRAZ6IEAFIUQIS3ZYOFMUTYWUQJGA, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.291467107 (408h49m9.162550152s) [noise=true , address=77.90.31.19:4844 , piece_id=P3ARVIRV6DHLSXYK2QBBSDPRCEAYLSHJLEAGIAJ6UYLBGSQRBK6A, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.292000592 (408h49m9.163083637s) [noise=true , address=145.79.227.48:4301 , piece_id=DAM6FOLQWAC2ZHA274NPGL743KI6QQUNPGQ5ETEPTFV2UCS2C4WA, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.292520292 (408h49m9.163603337s) [noise=true , address=145.79.227.42:4121 , piece_id=CRAOIQLWBTENWZN6AMDWSMKMXEAB36ZHELNL7737HALMZSPH3DSQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.293926140 (408h49m9.165009185s) [noise=true , address=145.79.227.41:4086 , piece_id=CIW56LRG4OSSKY7HDHKC352FN77KXADPWFJQ6AH44QWGRKCS2Y5Q, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.294123896 (408h49m9.165206931s) [noise=true , address=145.79.227.46:4602 , piece_id=SN22WFK7KRBZXRGQPVLRZ4CVRPU6Q34XIAX3Y4BJ4HSH3ZRWO5XQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.295190838 (408h49m9.166273873s) [noise=true , address=145.79.227.35:4482 , piece_id=PWJDZXJ2BWCNLDXIEHNOQHTDZ527TIBXIZQWDV7QR4G6K5PM5GFQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.295672685 (408h49m9.16675573s) [noise=true , address=145.79.227.8:4329 , piece_id=PIMLYVB45QTX3RKSSF5HVO7DVWSTEDBPUPEI4YYDKL55OK3G7GFA, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.296512795 (408h49m9.16759583s) [noise=true , address=145.79.227.14:4837 , piece_id=UGUTYFGPK3WF3N3DMBWHQHBNWSU7DXHXSPVTUIITKCMY2M7PFR6A, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.297667263 (408h49m9.168750298s) [noise=true , address=145.79.227.47:4265 , piece_id=5R37XFDZNHGQYJ2FINHLAHDB4MXJWMAYNB2HYHO4B5UM4ETORBZQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.298831309 (408h49m9.169914344s) [noise=true , address=145.79.227.37:4566 , piece_id=C4XIYBXUXTGL6AQMNSIU2SCC6TZNV54NYQLAOYVEGHFBFOO7S3LQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.300152595 (408h49m9.17123564s) [noise=true , address=145.79.227.50:4375 , piece_id=UDPHQP42KHKH3CBIAILJM6BJ34UAA2D7GA3BIXDK5IOYBLWHO7YQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.301524037 (408h49m9.172607072s) [noise=true , address=145.79.227.32:4835 , piece_id=6XVWDEHT4KKJUQMXFFVJU7EGZTUXIHFGRXC2CI66ZA3ERBJXEMVA, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:11.170954938 (408h49m11.042037983s) [upload=358898]: Next returned nil segment Dec 22 16:18:11.170982821 (408h49m11.042065866s) [upload=358898]: Waiting for error group managing segments... Dec 22 16:18:11.470411986 (408h49m11.341495021s) [noise=true , address=77.90.31.14:4662 , piece_id=CQSKEBWWSNDMYXCVCYE6HKVEJ4BXSM54ABTDTLLDFPSBZB34BI7Q, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:11.514534372 (408h49m11.385617417s) [noise=true , address=45.91.116.14:4191 , piece_id=TYNCODGLLAGFMNAGPN6X7HJKRAZ6IEAFIUQIS3ZYOFMUTYWUQJGA, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:11.590140129 (408h49m11.461223174s) [noise=true , address=77.90.31.19:4844 , piece_id=P3ARVIRV6DHLSXYK2QBBSDPRCEAYLSHJLEAGIAJ6UYLBGSQRBK6A, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.029558086 (408h49m11.900641131s) [noise=true , address=145.79.227.48:4301 , piece_id=DAM6FOLQWAC2ZHA274NPGL743KI6QQUNPGQ5ETEPTFV2UCS2C4WA, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.050418598 (408h49m11.921501633s) [noise=true , address=145.79.227.8:4329 , piece_id=PIMLYVB45QTX3RKSSF5HVO7DVWSTEDBPUPEI4YYDKL55OK3G7GFA, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.063480270 (408h49m11.934563326s) [noise=true , address=145.79.227.47:4265 , piece_id=5R37XFDZNHGQYJ2FINHLAHDB4MXJWMAYNB2HYHO4B5UM4ETORBZQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.063617061 (408h49m11.934700106s) [noise=true , address=145.79.227.35:4482 , piece_id=PWJDZXJ2BWCNLDXIEHNOQHTDZ527TIBXIZQWDV7QR4G6K5PM5GFQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.075770333 (408h49m11.946853378s) [noise=true , address=145.79.227.46:4602 , piece_id=SN22WFK7KRBZXRGQPVLRZ4CVRPU6Q34XIAX3Y4BJ4HSH3ZRWO5XQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.114183588 (408h49m11.985266653s) [noise=true , address=145.79.227.37:4566 , piece_id=C4XIYBXUXTGL6AQMNSIU2SCC6TZNV54NYQLAOYVEGHFBFOO7S3LQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.128980844 (408h49m12.000063879s) [noise=true , address=145.79.227.50:4375 , piece_id=UDPHQP42KHKH3CBIAILJM6BJ34UAA2D7GA3BIXDK5IOYBLWHO7YQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.151198050 (408h49m12.022281096s) [noise=true , address=145.79.227.13:4803 , piece_id=NS57HINS2HYB56GFXJWNFCVB6BOA7AA5MTHOW2YVLFK3PTPDF5HQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.156165298 (408h49m12.027248343s) [noise=true , address=145.79.227.42:4121 , piece_id=CRAOIQLWBTENWZN6AMDWSMKMXEAB36ZHELNL7737HALMZSPH3DSQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.156486050 (408h49m12.027569095s) [noise=true , address=145.79.227.41:4086 , piece_id=CIW56LRG4OSSKY7HDHKC352FN77KXADPWFJQ6AH44QWGRKCS2Y5Q, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.156512820 (408h49m12.027595865s) [seg_pos={0 0}, upload=358898]: Segment reached optimal threshold of 13 pieces. Dec 22 16:18:12.156773717 (408h49m12.027856762s) [noise=true , address=145.79.227.14:4837 , piece_id=UGUTYFGPK3WF3N3DMBWHQHBNWSU7DXHXSPVTUIITKCMY2M7PFR6A, seg_pos={0 0}, upload=358898]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: expected piece hash; context canceled; EOF Dec 22 16:18:12.156786501 (408h49m12.027869546s) [noise=true , address=145.79.227.14:4837 , piece_id=UGUTYFGPK3WF3N3DMBWHQHBNWSU7DXHXSPVTUIITKCMY2M7PFR6A, seg_pos={0 0}, upload=358898]: Long tail cancellation due to optimal threshold Dec 22 16:18:12.156853008 (408h49m12.027936063s) [noise=true , address=145.79.227.32:4835 , piece_id=6XVWDEHT4KKJUQMXFFVJU7EGZTUXIHFGRXC2CI66ZA3ERBJXEMVA, seg_pos={0 0}, upload=358898]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y): protocol: expected piece hash; context canceled; EOF Dec 22 16:18:12.156864379 (408h49m12.027947434s) [noise=true , address=145.79.227.32:4835 , piece_id=6XVWDEHT4KKJUQMXFFVJU7EGZTUXIHFGRXC2CI66ZA3ERBJXEMVA, seg_pos={0 0}, upload=358898]: Long tail cancellation due to optimal threshold Dec 22 16:18:12.157100860 (408h49m12.028183905s) [noise=true , address=145.79.227.51:4409 , piece_id=GFDYLOEWMOIO2MSAZDXIOBXR6P2DSEAX56CDF4T7SLGPXS3AO6BA, seg_pos={0 0}, upload=358898]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: expected piece hash; context canceled; EOF Dec 22 16:18:12.157115689 (408h49m12.028198754s) [noise=true , address=145.79.227.51:4409 , piece_id=GFDYLOEWMOIO2MSAZDXIOBXR6P2DSEAX56CDF4T7SLGPXS3AO6BA, seg_pos={0 0}, upload=358898]: Long tail cancellation due to optimal threshold Dec 22 16:18:12.157144382 (408h49m12.028227418s) [seg_pos={0 0}, upload=358898]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 22 16:18:12.157297575 (408h49m12.02838063s) [upload=358898]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 16:18:12.157310810 (408h49m12.028393855s) [upload=358898]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 16:18:12.458231164 (408h49m12.32931423s) [upload=358898]: Done uploading segments... Dec 22 16:18:12.458268646 (408h49m12.329351691s) [upload=358898]: Upload finished. err: <nil> Duration: 8.740161358s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 1.364354906s trace: https://jaeger.dev.storj.io/trace/6ea3fd74aff47917 FAIL (2025-12-22 16:19:47 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.364390555s core uplink eu1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 476.432691ms target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/704b2d4a066e7708 FAIL (2025-12-22 16:19:42 UTC) orbiter warning: Get "https://eu1.storj.io/login": dial tcp 35.207.121.91:443: i/o timeout Duration: 476.459341ms core satellite eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 1.396239893s trace: https://jaeger.dev.storj.io/trace/6ea3fd74aff47917 FAIL (2025-12-22 16:19:24 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.396267836s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............X............................X................] duration: 5.525749391s trace: https://jaeger.dev.storj.io/trace/72726d8466c916ea FAIL (2025-12-22 16:19:04 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 5.525780341s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 1.494092881s trace: https://jaeger.dev.storj.io/trace/6ea3fd74aff47917 FAIL (2025-12-22 16:19:03 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.494149449s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [....XXX..X...X..XXXX....X..X...XX.....XXXX......X.X.X.X...X.] duration: 13.013464827s trace: https://jaeger.dev.storj.io/trace/6eb5416200619b22 FAIL (2025-12-22 16:18:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 22 16:18:09.037402914 (408h49m8.908485959s) [upload=358898]: Starting upload Dec 22 16:18:09.037510439 (408h49m8.908593504s) [upload=358898]: Done starting upload Dec 22 16:18:09.037545956 (408h49m8.908629002s) [upload=358898]: Uploading segments... Dec 22 16:18:09.037759613 (408h49m8.908842658s) [upload=358898]: Got next segment. Inline: false Dec 22 16:18:09.037807204 (408h49m8.90889025s) [upload=358898]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 16:18:09.037813126 (408h49m8.908896171s) [upload=358898]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 16:18:09.289696365 (408h49m9.1607794s) [seg_pos={0 0}, upload=358898]: Begin upload segment... Dec 22 16:18:09.289741130 (408h49m9.160824175s) [seg_pos={0 0}, upload=358898]: Done begin upload segment. Dec 22 16:18:09.289767701 (408h49m9.160850746s) [noise=true , address=145.79.227.13:4803 , piece_id=NS57HINS2HYB56GFXJWNFCVB6BOA7AA5MTHOW2YVLFK3PTPDF5HQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.289986888 (408h49m9.161069923s) [noise=true , address=77.90.31.14:4662 , piece_id=CQSKEBWWSNDMYXCVCYE6HKVEJ4BXSM54ABTDTLLDFPSBZB34BI7Q, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.290568395 (408h49m9.16165144s) [noise=true , address=145.79.227.51:4409 , piece_id=GFDYLOEWMOIO2MSAZDXIOBXR6P2DSEAX56CDF4T7SLGPXS3AO6BA, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.291017320 (408h49m9.162100355s) [noise=true , address=45.91.116.14:4191 , piece_id=TYNCODGLLAGFMNAGPN6X7HJKRAZ6IEAFIUQIS3ZYOFMUTYWUQJGA, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.291467107 (408h49m9.162550152s) [noise=true , address=77.90.31.19:4844 , piece_id=P3ARVIRV6DHLSXYK2QBBSDPRCEAYLSHJLEAGIAJ6UYLBGSQRBK6A, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.292000592 (408h49m9.163083637s) [noise=true , address=145.79.227.48:4301 , piece_id=DAM6FOLQWAC2ZHA274NPGL743KI6QQUNPGQ5ETEPTFV2UCS2C4WA, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.292520292 (408h49m9.163603337s) [noise=true , address=145.79.227.42:4121 , piece_id=CRAOIQLWBTENWZN6AMDWSMKMXEAB36ZHELNL7737HALMZSPH3DSQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.293926140 (408h49m9.165009185s) [noise=true , address=145.79.227.41:4086 , piece_id=CIW56LRG4OSSKY7HDHKC352FN77KXADPWFJQ6AH44QWGRKCS2Y5Q, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.294123896 (408h49m9.165206931s) [noise=true , address=145.79.227.46:4602 , piece_id=SN22WFK7KRBZXRGQPVLRZ4CVRPU6Q34XIAX3Y4BJ4HSH3ZRWO5XQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.295190838 (408h49m9.166273873s) [noise=true , address=145.79.227.35:4482 , piece_id=PWJDZXJ2BWCNLDXIEHNOQHTDZ527TIBXIZQWDV7QR4G6K5PM5GFQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.295672685 (408h49m9.16675573s) [noise=true , address=145.79.227.8:4329 , piece_id=PIMLYVB45QTX3RKSSF5HVO7DVWSTEDBPUPEI4YYDKL55OK3G7GFA, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.296512795 (408h49m9.16759583s) [noise=true , address=145.79.227.14:4837 , piece_id=UGUTYFGPK3WF3N3DMBWHQHBNWSU7DXHXSPVTUIITKCMY2M7PFR6A, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.297667263 (408h49m9.168750298s) [noise=true , address=145.79.227.47:4265 , piece_id=5R37XFDZNHGQYJ2FINHLAHDB4MXJWMAYNB2HYHO4B5UM4ETORBZQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.298831309 (408h49m9.169914344s) [noise=true , address=145.79.227.37:4566 , piece_id=C4XIYBXUXTGL6AQMNSIU2SCC6TZNV54NYQLAOYVEGHFBFOO7S3LQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.300152595 (408h49m9.17123564s) [noise=true , address=145.79.227.50:4375 , piece_id=UDPHQP42KHKH3CBIAILJM6BJ34UAA2D7GA3BIXDK5IOYBLWHO7YQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.301524037 (408h49m9.172607072s) [noise=true , address=145.79.227.32:4835 , piece_id=6XVWDEHT4KKJUQMXFFVJU7EGZTUXIHFGRXC2CI66ZA3ERBJXEMVA, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:11.170954938 (408h49m11.042037983s) [upload=358898]: Next returned nil segment Dec 22 16:18:11.170982821 (408h49m11.042065866s) [upload=358898]: Waiting for error group managing segments... Dec 22 16:18:11.470411986 (408h49m11.341495021s) [noise=true , address=77.90.31.14:4662 , piece_id=CQSKEBWWSNDMYXCVCYE6HKVEJ4BXSM54ABTDTLLDFPSBZB34BI7Q, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:11.514534372 (408h49m11.385617417s) [noise=true , address=45.91.116.14:4191 , piece_id=TYNCODGLLAGFMNAGPN6X7HJKRAZ6IEAFIUQIS3ZYOFMUTYWUQJGA, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:11.590140129 (408h49m11.461223174s) [noise=true , address=77.90.31.19:4844 , piece_id=P3ARVIRV6DHLSXYK2QBBSDPRCEAYLSHJLEAGIAJ6UYLBGSQRBK6A, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.029558086 (408h49m11.900641131s) [noise=true , address=145.79.227.48:4301 , piece_id=DAM6FOLQWAC2ZHA274NPGL743KI6QQUNPGQ5ETEPTFV2UCS2C4WA, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.050418598 (408h49m11.921501633s) [noise=true , address=145.79.227.8:4329 , piece_id=PIMLYVB45QTX3RKSSF5HVO7DVWSTEDBPUPEI4YYDKL55OK3G7GFA, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.063480270 (408h49m11.934563326s) [noise=true , address=145.79.227.47:4265 , piece_id=5R37XFDZNHGQYJ2FINHLAHDB4MXJWMAYNB2HYHO4B5UM4ETORBZQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.063617061 (408h49m11.934700106s) [noise=true , address=145.79.227.35:4482 , piece_id=PWJDZXJ2BWCNLDXIEHNOQHTDZ527TIBXIZQWDV7QR4G6K5PM5GFQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.075770333 (408h49m11.946853378s) [noise=true , address=145.79.227.46:4602 , piece_id=SN22WFK7KRBZXRGQPVLRZ4CVRPU6Q34XIAX3Y4BJ4HSH3ZRWO5XQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.114183588 (408h49m11.985266653s) [noise=true , address=145.79.227.37:4566 , piece_id=C4XIYBXUXTGL6AQMNSIU2SCC6TZNV54NYQLAOYVEGHFBFOO7S3LQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.128980844 (408h49m12.000063879s) [noise=true , address=145.79.227.50:4375 , piece_id=UDPHQP42KHKH3CBIAILJM6BJ34UAA2D7GA3BIXDK5IOYBLWHO7YQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.151198050 (408h49m12.022281096s) [noise=true , address=145.79.227.13:4803 , piece_id=NS57HINS2HYB56GFXJWNFCVB6BOA7AA5MTHOW2YVLFK3PTPDF5HQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.156165298 (408h49m12.027248343s) [noise=true , address=145.79.227.42:4121 , piece_id=CRAOIQLWBTENWZN6AMDWSMKMXEAB36ZHELNL7737HALMZSPH3DSQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.156486050 (408h49m12.027569095s) [noise=true , address=145.79.227.41:4086 , piece_id=CIW56LRG4OSSKY7HDHKC352FN77KXADPWFJQ6AH44QWGRKCS2Y5Q, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.156512820 (408h49m12.027595865s) [seg_pos={0 0}, upload=358898]: Segment reached optimal threshold of 13 pieces. Dec 22 16:18:12.156773717 (408h49m12.027856762s) [noise=true , address=145.79.227.14:4837 , piece_id=UGUTYFGPK3WF3N3DMBWHQHBNWSU7DXHXSPVTUIITKCMY2M7PFR6A, seg_pos={0 0}, upload=358898]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: expected piece hash; context canceled; EOF Dec 22 16:18:12.156786501 (408h49m12.027869546s) [noise=true , address=145.79.227.14:4837 , piece_id=UGUTYFGPK3WF3N3DMBWHQHBNWSU7DXHXSPVTUIITKCMY2M7PFR6A, seg_pos={0 0}, upload=358898]: Long tail cancellation due to optimal threshold Dec 22 16:18:12.156853008 (408h49m12.027936063s) [noise=true , address=145.79.227.32:4835 , piece_id=6XVWDEHT4KKJUQMXFFVJU7EGZTUXIHFGRXC2CI66ZA3ERBJXEMVA, seg_pos={0 0}, upload=358898]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y): protocol: expected piece hash; context canceled; EOF Dec 22 16:18:12.156864379 (408h49m12.027947434s) [noise=true , address=145.79.227.32:4835 , piece_id=6XVWDEHT4KKJUQMXFFVJU7EGZTUXIHFGRXC2CI66ZA3ERBJXEMVA, seg_pos={0 0}, upload=358898]: Long tail cancellation due to optimal threshold Dec 22 16:18:12.157100860 (408h49m12.028183905s) [noise=true , address=145.79.227.51:4409 , piece_id=GFDYLOEWMOIO2MSAZDXIOBXR6P2DSEAX56CDF4T7SLGPXS3AO6BA, seg_pos={0 0}, upload=358898]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: expected piece hash; context canceled; EOF Dec 22 16:18:12.157115689 (408h49m12.028198754s) [noise=true , address=145.79.227.51:4409 , piece_id=GFDYLOEWMOIO2MSAZDXIOBXR6P2DSEAX56CDF4T7SLGPXS3AO6BA, seg_pos={0 0}, upload=358898]: Long tail cancellation due to optimal threshold Dec 22 16:18:12.157144382 (408h49m12.028227418s) [seg_pos={0 0}, upload=358898]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 22 16:18:12.157297575 (408h49m12.02838063s) [upload=358898]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 16:18:12.157310810 (408h49m12.028393855s) [upload=358898]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 16:18:12.458231164 (408h49m12.32931423s) [upload=358898]: Done uploading segments... Dec 22 16:18:12.458268646 (408h49m12.329351691s) [upload=358898]: Upload finished. err: <nil> Duration: 13.013492138s core uplink us-select-1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 460.700834ms target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/704b2d4a066e7708 FAIL (2025-12-22 16:18:40 UTC) orbiter warning: Get "https://eu1.storj.io/login": dial tcp 35.207.121.91:443: i/o timeout Duration: 460.731101ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [X....XXX..X...X..XXXX....X..X...XX.....XXXX......X.X.X.X...X] duration: 30.002720549s trace: https://jaeger.dev.storj.io/trace/6eb5416200619b22 FAIL (2025-12-22 16:18:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 22 16:18:09.037402914 (408h49m8.908485959s) [upload=358898]: Starting upload Dec 22 16:18:09.037510439 (408h49m8.908593504s) [upload=358898]: Done starting upload Dec 22 16:18:09.037545956 (408h49m8.908629002s) [upload=358898]: Uploading segments... Dec 22 16:18:09.037759613 (408h49m8.908842658s) [upload=358898]: Got next segment. Inline: false Dec 22 16:18:09.037807204 (408h49m8.90889025s) [upload=358898]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 16:18:09.037813126 (408h49m8.908896171s) [upload=358898]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 16:18:09.289696365 (408h49m9.1607794s) [seg_pos={0 0}, upload=358898]: Begin upload segment... Dec 22 16:18:09.289741130 (408h49m9.160824175s) [seg_pos={0 0}, upload=358898]: Done begin upload segment. Dec 22 16:18:09.289767701 (408h49m9.160850746s) [noise=true , address=145.79.227.13:4803 , piece_id=NS57HINS2HYB56GFXJWNFCVB6BOA7AA5MTHOW2YVLFK3PTPDF5HQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.289986888 (408h49m9.161069923s) [noise=true , address=77.90.31.14:4662 , piece_id=CQSKEBWWSNDMYXCVCYE6HKVEJ4BXSM54ABTDTLLDFPSBZB34BI7Q, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.290568395 (408h49m9.16165144s) [noise=true , address=145.79.227.51:4409 , piece_id=GFDYLOEWMOIO2MSAZDXIOBXR6P2DSEAX56CDF4T7SLGPXS3AO6BA, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.291017320 (408h49m9.162100355s) [noise=true , address=45.91.116.14:4191 , piece_id=TYNCODGLLAGFMNAGPN6X7HJKRAZ6IEAFIUQIS3ZYOFMUTYWUQJGA, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.291467107 (408h49m9.162550152s) [noise=true , address=77.90.31.19:4844 , piece_id=P3ARVIRV6DHLSXYK2QBBSDPRCEAYLSHJLEAGIAJ6UYLBGSQRBK6A, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.292000592 (408h49m9.163083637s) [noise=true , address=145.79.227.48:4301 , piece_id=DAM6FOLQWAC2ZHA274NPGL743KI6QQUNPGQ5ETEPTFV2UCS2C4WA, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.292520292 (408h49m9.163603337s) [noise=true , address=145.79.227.42:4121 , piece_id=CRAOIQLWBTENWZN6AMDWSMKMXEAB36ZHELNL7737HALMZSPH3DSQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.293926140 (408h49m9.165009185s) [noise=true , address=145.79.227.41:4086 , piece_id=CIW56LRG4OSSKY7HDHKC352FN77KXADPWFJQ6AH44QWGRKCS2Y5Q, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.294123896 (408h49m9.165206931s) [noise=true , address=145.79.227.46:4602 , piece_id=SN22WFK7KRBZXRGQPVLRZ4CVRPU6Q34XIAX3Y4BJ4HSH3ZRWO5XQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.295190838 (408h49m9.166273873s) [noise=true , address=145.79.227.35:4482 , piece_id=PWJDZXJ2BWCNLDXIEHNOQHTDZ527TIBXIZQWDV7QR4G6K5PM5GFQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.295672685 (408h49m9.16675573s) [noise=true , address=145.79.227.8:4329 , piece_id=PIMLYVB45QTX3RKSSF5HVO7DVWSTEDBPUPEI4YYDKL55OK3G7GFA, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.296512795 (408h49m9.16759583s) [noise=true , address=145.79.227.14:4837 , piece_id=UGUTYFGPK3WF3N3DMBWHQHBNWSU7DXHXSPVTUIITKCMY2M7PFR6A, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.297667263 (408h49m9.168750298s) [noise=true , address=145.79.227.47:4265 , piece_id=5R37XFDZNHGQYJ2FINHLAHDB4MXJWMAYNB2HYHO4B5UM4ETORBZQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.298831309 (408h49m9.169914344s) [noise=true , address=145.79.227.37:4566 , piece_id=C4XIYBXUXTGL6AQMNSIU2SCC6TZNV54NYQLAOYVEGHFBFOO7S3LQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.300152595 (408h49m9.17123564s) [noise=true , address=145.79.227.50:4375 , piece_id=UDPHQP42KHKH3CBIAILJM6BJ34UAA2D7GA3BIXDK5IOYBLWHO7YQ, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:09.301524037 (408h49m9.172607072s) [noise=true , address=145.79.227.32:4835 , piece_id=6XVWDEHT4KKJUQMXFFVJU7EGZTUXIHFGRXC2CI66ZA3ERBJXEMVA, seg_pos={0 0}, upload=358898]: Uploading piece... Dec 22 16:18:11.170954938 (408h49m11.042037983s) [upload=358898]: Next returned nil segment Dec 22 16:18:11.170982821 (408h49m11.042065866s) [upload=358898]: Waiting for error group managing segments... Dec 22 16:18:11.470411986 (408h49m11.341495021s) [noise=true , address=77.90.31.14:4662 , piece_id=CQSKEBWWSNDMYXCVCYE6HKVEJ4BXSM54ABTDTLLDFPSBZB34BI7Q, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:11.514534372 (408h49m11.385617417s) [noise=true , address=45.91.116.14:4191 , piece_id=TYNCODGLLAGFMNAGPN6X7HJKRAZ6IEAFIUQIS3ZYOFMUTYWUQJGA, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:11.590140129 (408h49m11.461223174s) [noise=true , address=77.90.31.19:4844 , piece_id=P3ARVIRV6DHLSXYK2QBBSDPRCEAYLSHJLEAGIAJ6UYLBGSQRBK6A, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.029558086 (408h49m11.900641131s) [noise=true , address=145.79.227.48:4301 , piece_id=DAM6FOLQWAC2ZHA274NPGL743KI6QQUNPGQ5ETEPTFV2UCS2C4WA, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.050418598 (408h49m11.921501633s) [noise=true , address=145.79.227.8:4329 , piece_id=PIMLYVB45QTX3RKSSF5HVO7DVWSTEDBPUPEI4YYDKL55OK3G7GFA, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.063480270 (408h49m11.934563326s) [noise=true , address=145.79.227.47:4265 , piece_id=5R37XFDZNHGQYJ2FINHLAHDB4MXJWMAYNB2HYHO4B5UM4ETORBZQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.063617061 (408h49m11.934700106s) [noise=true , address=145.79.227.35:4482 , piece_id=PWJDZXJ2BWCNLDXIEHNOQHTDZ527TIBXIZQWDV7QR4G6K5PM5GFQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.075770333 (408h49m11.946853378s) [noise=true , address=145.79.227.46:4602 , piece_id=SN22WFK7KRBZXRGQPVLRZ4CVRPU6Q34XIAX3Y4BJ4HSH3ZRWO5XQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.114183588 (408h49m11.985266653s) [noise=true , address=145.79.227.37:4566 , piece_id=C4XIYBXUXTGL6AQMNSIU2SCC6TZNV54NYQLAOYVEGHFBFOO7S3LQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.128980844 (408h49m12.000063879s) [noise=true , address=145.79.227.50:4375 , piece_id=UDPHQP42KHKH3CBIAILJM6BJ34UAA2D7GA3BIXDK5IOYBLWHO7YQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.151198050 (408h49m12.022281096s) [noise=true , address=145.79.227.13:4803 , piece_id=NS57HINS2HYB56GFXJWNFCVB6BOA7AA5MTHOW2YVLFK3PTPDF5HQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.156165298 (408h49m12.027248343s) [noise=true , address=145.79.227.42:4121 , piece_id=CRAOIQLWBTENWZN6AMDWSMKMXEAB36ZHELNL7737HALMZSPH3DSQ, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.156486050 (408h49m12.027569095s) [noise=true , address=145.79.227.41:4086 , piece_id=CIW56LRG4OSSKY7HDHKC352FN77KXADPWFJQ6AH44QWGRKCS2Y5Q, seg_pos={0 0}, upload=358898]: Done uploading piece. err: <nil> Dec 22 16:18:12.156512820 (408h49m12.027595865s) [seg_pos={0 0}, upload=358898]: Segment reached optimal threshold of 13 pieces. Dec 22 16:18:12.156773717 (408h49m12.027856762s) [noise=true , address=145.79.227.14:4837 , piece_id=UGUTYFGPK3WF3N3DMBWHQHBNWSU7DXHXSPVTUIITKCMY2M7PFR6A, seg_pos={0 0}, upload=358898]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: expected piece hash; context canceled; EOF Dec 22 16:18:12.156786501 (408h49m12.027869546s) [noise=true , address=145.79.227.14:4837 , piece_id=UGUTYFGPK3WF3N3DMBWHQHBNWSU7DXHXSPVTUIITKCMY2M7PFR6A, seg_pos={0 0}, upload=358898]: Long tail cancellation due to optimal threshold Dec 22 16:18:12.156853008 (408h49m12.027936063s) [noise=true , address=145.79.227.32:4835 , piece_id=6XVWDEHT4KKJUQMXFFVJU7EGZTUXIHFGRXC2CI66ZA3ERBJXEMVA, seg_pos={0 0}, upload=358898]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16vVfeoqCkbMYCRjMMWaKBnCvQBBJPMgwQtWTq9HKQrD9SS17y): protocol: expected piece hash; context canceled; EOF Dec 22 16:18:12.156864379 (408h49m12.027947434s) [noise=true , address=145.79.227.32:4835 , piece_id=6XVWDEHT4KKJUQMXFFVJU7EGZTUXIHFGRXC2CI66ZA3ERBJXEMVA, seg_pos={0 0}, upload=358898]: Long tail cancellation due to optimal threshold Dec 22 16:18:12.157100860 (408h49m12.028183905s) [noise=true , address=145.79.227.51:4409 , piece_id=GFDYLOEWMOIO2MSAZDXIOBXR6P2DSEAX56CDF4T7SLGPXS3AO6BA, seg_pos={0 0}, upload=358898]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: expected piece hash; context canceled; EOF Dec 22 16:18:12.157115689 (408h49m12.028198754s) [noise=true , address=145.79.227.51:4409 , piece_id=GFDYLOEWMOIO2MSAZDXIOBXR6P2DSEAX56CDF4T7SLGPXS3AO6BA, seg_pos={0 0}, upload=358898]: Long tail cancellation due to optimal threshold Dec 22 16:18:12.157144382 (408h49m12.028227418s) [seg_pos={0 0}, upload=358898]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 22 16:18:12.157297575 (408h49m12.02838063s) [upload=358898]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 16:18:12.157310810 (408h49m12.028393855s) [upload=358898]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 16:18:12.458231164 (408h49m12.32931423s) [upload=358898]: Done uploading segments... Dec 22 16:18:12.458268646 (408h49m12.329351691s) [upload=358898]: Upload finished. err: <nil> Duration: 30.002743312s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............X............................X...............] duration: 6.082143763s trace: https://jaeger.dev.storj.io/trace/72726d8466c916ea FAIL (2025-12-22 16:18:27 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.082172617s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 1.375381774s trace: https://jaeger.dev.storj.io/trace/6ea3fd74aff47917 FAIL (2025-12-22 16:18:14 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.375407603s core uplink eu1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 468.755231ms target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/704b2d4a066e7708 FAIL (2025-12-22 16:18:03 UTC) orbiter warning: Get "https://eu1.storj.io/login": dial tcp 35.207.121.91:443: i/o timeout Duration: 468.810027ms core satellite eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................X............................X..............] duration: 5.532582854s trace: https://jaeger.dev.storj.io/trace/72726d8466c916ea FAIL (2025-12-22 16:17:28 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 5.532610718s core uplink eu1

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)