Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.297818061s trace: https://jaeger.dev.storj.io/trace/3c39be5f434273aa FAIL (2025-12-23 09:10:29 UTC) orbiter warning: context deadline exceeded Duration: 6.297838229s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.87914503s trace: https://jaeger.dev.storj.io/trace/fbd33bd94283fbe FAIL (2025-12-23 09:10:22 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.879175208s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 26.078926333s trace: https://jaeger.dev.storj.io/trace/3b621a03316dcb85 FAIL (2025-12-23 09:10:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1NgPjVCyZZAfxVr3WvKrGRjnkpPtUvmQcCMiYntxNKVSWqXcHC, Piece ID: MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ) context deadline exceeded; piecestore: (Node ID: 14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj, Piece ID: SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ) context deadline exceeded; piecestore: (Node ID: 124taxpXdSnzEkq6UG1ZPi7cKLkaoDSc3G9FH1YHbGbDwMTM8PG, Piece ID: XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA) context deadline exceeded; piecestore: (Node ID: 121nTAu2qwnGorPfRFRJ7kE4hSsPq9p8zCZVFtt7cNNGu1HGdnN, Piece ID: KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA) context deadline exceeded ======LOGS====== 3358]: Uploading piece... Dec 23 08:47:14.746900773 (425h18m14.617983818s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.749939840 (425h18m14.621022885s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.753510229 (425h18m14.624593264s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.755262356 (425h18m14.626345391s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.756625842 (425h18m14.627708877s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.758025117 (425h18m14.629108172s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.760642110 (425h18m14.631725145s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.761429270 (425h18m14.632512315s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763241921 (425h18m14.634324966s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763973835 (425h18m14.63505688s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765312685 (425h18m14.63639573s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765648824 (425h18m14.636731869s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.767253601 (425h18m14.638336636s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.768200434 (425h18m14.639283469s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:16.245896991 (425h18m16.116980036s) [upload=373358]: Next returned nil segment Dec 23 08:47:16.245944181 (425h18m16.117027226s) [upload=373358]: Waiting for error group managing segments... Dec 23 08:47:16.644138110 (425h18m16.515221145s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.131664201 (425h18m17.002747236s) [noise=true , address=64.49.13.139:4475 , piece_id=TH3UWCMUNGGADRA26PZNKZLOWCLRLRWIT23ONF2B2DJ3NMMFPZBQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.444602902 (425h18m17.315685947s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.470421138 (425h18m17.341504173s) [noise=true , address=77.90.31.13:4630 , piece_id=SOJYUZH676PZYM4Z5YGEXW4ZXPAVLSEIFFZBE22LGAUOMRRKVG7A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.554527268 (425h18m17.425610313s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.059191355 (425h18m17.93027439s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.354549959 (425h18m18.225633004s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.436262252 (425h18m18.307345297s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.800612455 (425h18m18.67169549s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.973704980 (425h18m18.844788025s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.276001403 (425h18m19.147084438s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.339083580 (425h18m19.210166625s) [noise=true , address=77.90.31.19:4876 , piece_id=EO3PO3DW5CXWWO4T6ND7J4JTZYC3RI35MLMUEQXQVEABP63LQMQQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.349762695 (425h18m19.22084573s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636789835 (425h18m20.50787288s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636817878 (425h18m20.507900923s) [seg_pos={0 0}, upload=373358]: Segment reached optimal threshold of 14 pieces. Dec 23 08:47:20.637067674 (425h18m20.508150709s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf): protocol: expected piece hash; context canceled; EOF Dec 23 08:47:20.637077883 (425h18m20.508160938s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637240834 (425h18m20.508323879s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu): protocol: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection Dec 23 08:47:20.637249079 (425h18m20.508332125s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637346505 (425h18m20.50842956s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bFEmP78qGhyHhXr3QNVptdiWQ4No4nrb31qj6DPgmJLcHYKiM): protocol: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection Dec 23 08:47:20.637368196 (425h18m20.508451241s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637562967 (425h18m20.508646012s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1acQG8MjLvgzmCs8ELswcsKJrNvYmGQLX1FL4dHeCj2AUH7A5R): protocol: read tcp 188.166.225.170:56478->45.91.116.13:4170: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 23 08:47:20.637572605 (425h18m20.50865564s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637715547 (425h18m20.508798592s) [seg_pos={0 0}, upload=373358]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 23 08:47:20.637871194 (425h18m20.508954239s) [upload=373358]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 08:47:20.637878588 (425h18m20.508961633s) [upload=373358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 08:47:21.234004555 (425h18m21.10508763s) [upload=373358]: Done uploading segments... Dec 23 08:47:21.234072815 (425h18m21.10515586s) [upload=373358]: Upload finished. err: <nil> Duration: 26.078944217s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 1.432666943s trace: https://jaeger.dev.storj.io/trace/292b9465c989993a FAIL (2025-12-23 09:10:07 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 1.432697272s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 1.943366366s trace: https://jaeger.dev.storj.io/trace/3727d7c571aed99f FAIL (2025-12-23 09:10:06 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.943439434s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.488062838s trace: https://jaeger.dev.storj.io/trace/fbd33bd94283fbe FAIL (2025-12-23 09:09:59 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 6.488081793s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.749388471s trace: https://jaeger.dev.storj.io/trace/3c39be5f434273aa FAIL (2025-12-23 09:09:57 UTC) orbiter warning: context deadline exceeded Duration: 5.749418698s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 20.080590871s trace: https://jaeger.dev.storj.io/trace/3b621a03316dcb85 FAIL (2025-12-23 09:09:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1NgPjVCyZZAfxVr3WvKrGRjnkpPtUvmQcCMiYntxNKVSWqXcHC, Piece ID: MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ) context deadline exceeded; piecestore: (Node ID: 14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj, Piece ID: SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ) context deadline exceeded; piecestore: (Node ID: 124taxpXdSnzEkq6UG1ZPi7cKLkaoDSc3G9FH1YHbGbDwMTM8PG, Piece ID: XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA) context deadline exceeded; piecestore: (Node ID: 121nTAu2qwnGorPfRFRJ7kE4hSsPq9p8zCZVFtt7cNNGu1HGdnN, Piece ID: KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA) context deadline exceeded ======LOGS====== 3358]: Uploading piece... Dec 23 08:47:14.746900773 (425h18m14.617983818s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.749939840 (425h18m14.621022885s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.753510229 (425h18m14.624593264s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.755262356 (425h18m14.626345391s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.756625842 (425h18m14.627708877s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.758025117 (425h18m14.629108172s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.760642110 (425h18m14.631725145s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.761429270 (425h18m14.632512315s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763241921 (425h18m14.634324966s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763973835 (425h18m14.63505688s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765312685 (425h18m14.63639573s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765648824 (425h18m14.636731869s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.767253601 (425h18m14.638336636s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.768200434 (425h18m14.639283469s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:16.245896991 (425h18m16.116980036s) [upload=373358]: Next returned nil segment Dec 23 08:47:16.245944181 (425h18m16.117027226s) [upload=373358]: Waiting for error group managing segments... Dec 23 08:47:16.644138110 (425h18m16.515221145s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.131664201 (425h18m17.002747236s) [noise=true , address=64.49.13.139:4475 , piece_id=TH3UWCMUNGGADRA26PZNKZLOWCLRLRWIT23ONF2B2DJ3NMMFPZBQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.444602902 (425h18m17.315685947s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.470421138 (425h18m17.341504173s) [noise=true , address=77.90.31.13:4630 , piece_id=SOJYUZH676PZYM4Z5YGEXW4ZXPAVLSEIFFZBE22LGAUOMRRKVG7A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.554527268 (425h18m17.425610313s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.059191355 (425h18m17.93027439s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.354549959 (425h18m18.225633004s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.436262252 (425h18m18.307345297s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.800612455 (425h18m18.67169549s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.973704980 (425h18m18.844788025s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.276001403 (425h18m19.147084438s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.339083580 (425h18m19.210166625s) [noise=true , address=77.90.31.19:4876 , piece_id=EO3PO3DW5CXWWO4T6ND7J4JTZYC3RI35MLMUEQXQVEABP63LQMQQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.349762695 (425h18m19.22084573s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636789835 (425h18m20.50787288s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636817878 (425h18m20.507900923s) [seg_pos={0 0}, upload=373358]: Segment reached optimal threshold of 14 pieces. Dec 23 08:47:20.637067674 (425h18m20.508150709s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf): protocol: expected piece hash; context canceled; EOF Dec 23 08:47:20.637077883 (425h18m20.508160938s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637240834 (425h18m20.508323879s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu): protocol: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection Dec 23 08:47:20.637249079 (425h18m20.508332125s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637346505 (425h18m20.50842956s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bFEmP78qGhyHhXr3QNVptdiWQ4No4nrb31qj6DPgmJLcHYKiM): protocol: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection Dec 23 08:47:20.637368196 (425h18m20.508451241s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637562967 (425h18m20.508646012s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1acQG8MjLvgzmCs8ELswcsKJrNvYmGQLX1FL4dHeCj2AUH7A5R): protocol: read tcp 188.166.225.170:56478->45.91.116.13:4170: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 23 08:47:20.637572605 (425h18m20.50865564s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637715547 (425h18m20.508798592s) [seg_pos={0 0}, upload=373358]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 23 08:47:20.637871194 (425h18m20.508954239s) [upload=373358]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 08:47:20.637878588 (425h18m20.508961633s) [upload=373358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 08:47:21.234004555 (425h18m21.10508763s) [upload=373358]: Done uploading segments... Dec 23 08:47:21.234072815 (425h18m21.10515586s) [upload=373358]: Upload finished. err: <nil> Duration: 20.080618465s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.987325894s trace: https://jaeger.dev.storj.io/trace/6156a7d944a046d FAIL (2025-12-23 09:09:40 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 6.987354127s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.154149108s trace: https://jaeger.dev.storj.io/trace/265f3259a7705b6d FAIL (2025-12-23 09:09:35 UTC) orbiter warning: operation: sync2.Copy uplink: metaclient: context deadline exceeded Duration: 5.154179706s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.979807247s trace: https://jaeger.dev.storj.io/trace/fbd33bd94283fbe FAIL (2025-12-23 09:09:33 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.979837756s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 1.409357437s trace: https://jaeger.dev.storj.io/trace/292b9465c989993a FAIL (2025-12-23 09:09:31 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 1.409381664s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 2.425105874s trace: https://jaeger.dev.storj.io/trace/3727d7c571aed99f FAIL (2025-12-23 09:09:26 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.425130059s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.170058582s trace: https://jaeger.dev.storj.io/trace/6156a7d944a046d FAIL (2025-12-23 09:09:18 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 6.1700946s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 4.86525902s trace: https://jaeger.dev.storj.io/trace/265f3259a7705b6d FAIL (2025-12-23 09:09:18 UTC) orbiter warning: operation: sync2.Copy uplink: metaclient: context deadline exceeded Duration: 4.865279941s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.978134753s trace: https://jaeger.dev.storj.io/trace/fbd33bd94283fbe FAIL (2025-12-23 09:09:17 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.978165861s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 21.686549217s trace: https://jaeger.dev.storj.io/trace/3b621a03316dcb85 FAIL (2025-12-23 09:09:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1NgPjVCyZZAfxVr3WvKrGRjnkpPtUvmQcCMiYntxNKVSWqXcHC, Piece ID: MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ) context deadline exceeded; piecestore: (Node ID: 14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj, Piece ID: SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ) context deadline exceeded; piecestore: (Node ID: 124taxpXdSnzEkq6UG1ZPi7cKLkaoDSc3G9FH1YHbGbDwMTM8PG, Piece ID: XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA) context deadline exceeded; piecestore: (Node ID: 121nTAu2qwnGorPfRFRJ7kE4hSsPq9p8zCZVFtt7cNNGu1HGdnN, Piece ID: KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA) context deadline exceeded ======LOGS====== 3358]: Uploading piece... Dec 23 08:47:14.746900773 (425h18m14.617983818s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.749939840 (425h18m14.621022885s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.753510229 (425h18m14.624593264s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.755262356 (425h18m14.626345391s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.756625842 (425h18m14.627708877s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.758025117 (425h18m14.629108172s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.760642110 (425h18m14.631725145s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.761429270 (425h18m14.632512315s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763241921 (425h18m14.634324966s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763973835 (425h18m14.63505688s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765312685 (425h18m14.63639573s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765648824 (425h18m14.636731869s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.767253601 (425h18m14.638336636s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.768200434 (425h18m14.639283469s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:16.245896991 (425h18m16.116980036s) [upload=373358]: Next returned nil segment Dec 23 08:47:16.245944181 (425h18m16.117027226s) [upload=373358]: Waiting for error group managing segments... Dec 23 08:47:16.644138110 (425h18m16.515221145s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.131664201 (425h18m17.002747236s) [noise=true , address=64.49.13.139:4475 , piece_id=TH3UWCMUNGGADRA26PZNKZLOWCLRLRWIT23ONF2B2DJ3NMMFPZBQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.444602902 (425h18m17.315685947s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.470421138 (425h18m17.341504173s) [noise=true , address=77.90.31.13:4630 , piece_id=SOJYUZH676PZYM4Z5YGEXW4ZXPAVLSEIFFZBE22LGAUOMRRKVG7A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.554527268 (425h18m17.425610313s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.059191355 (425h18m17.93027439s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.354549959 (425h18m18.225633004s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.436262252 (425h18m18.307345297s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.800612455 (425h18m18.67169549s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.973704980 (425h18m18.844788025s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.276001403 (425h18m19.147084438s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.339083580 (425h18m19.210166625s) [noise=true , address=77.90.31.19:4876 , piece_id=EO3PO3DW5CXWWO4T6ND7J4JTZYC3RI35MLMUEQXQVEABP63LQMQQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.349762695 (425h18m19.22084573s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636789835 (425h18m20.50787288s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636817878 (425h18m20.507900923s) [seg_pos={0 0}, upload=373358]: Segment reached optimal threshold of 14 pieces. Dec 23 08:47:20.637067674 (425h18m20.508150709s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf): protocol: expected piece hash; context canceled; EOF Dec 23 08:47:20.637077883 (425h18m20.508160938s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637240834 (425h18m20.508323879s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu): protocol: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection Dec 23 08:47:20.637249079 (425h18m20.508332125s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637346505 (425h18m20.50842956s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bFEmP78qGhyHhXr3QNVptdiWQ4No4nrb31qj6DPgmJLcHYKiM): protocol: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection Dec 23 08:47:20.637368196 (425h18m20.508451241s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637562967 (425h18m20.508646012s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1acQG8MjLvgzmCs8ELswcsKJrNvYmGQLX1FL4dHeCj2AUH7A5R): protocol: read tcp 188.166.225.170:56478->45.91.116.13:4170: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 23 08:47:20.637572605 (425h18m20.50865564s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637715547 (425h18m20.508798592s) [seg_pos={0 0}, upload=373358]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 23 08:47:20.637871194 (425h18m20.508954239s) [upload=373358]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 08:47:20.637878588 (425h18m20.508961633s) [upload=373358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 08:47:21.234004555 (425h18m21.10508763s) [upload=373358]: Done uploading segments... Dec 23 08:47:21.234072815 (425h18m21.10515586s) [upload=373358]: Upload finished. err: <nil> Duration: 21.686578403s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 6.393588172s trace: https://jaeger.dev.storj.io/trace/6156a7d944a046d FAIL (2025-12-23 09:08:55 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 6.393616034s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.030516507s trace: https://jaeger.dev.storj.io/trace/3c39be5f434273aa FAIL (2025-12-23 09:08:51 UTC) orbiter warning: context deadline exceeded Duration: 6.030547026s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 18.062816411s trace: https://jaeger.dev.storj.io/trace/3b621a03316dcb85 FAIL (2025-12-23 09:08:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1NgPjVCyZZAfxVr3WvKrGRjnkpPtUvmQcCMiYntxNKVSWqXcHC, Piece ID: MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ) context deadline exceeded; piecestore: (Node ID: 14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj, Piece ID: SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ) context deadline exceeded; piecestore: (Node ID: 124taxpXdSnzEkq6UG1ZPi7cKLkaoDSc3G9FH1YHbGbDwMTM8PG, Piece ID: XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA) context deadline exceeded; piecestore: (Node ID: 121nTAu2qwnGorPfRFRJ7kE4hSsPq9p8zCZVFtt7cNNGu1HGdnN, Piece ID: KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA) context deadline exceeded ======LOGS====== 3358]: Uploading piece... Dec 23 08:47:14.746900773 (425h18m14.617983818s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.749939840 (425h18m14.621022885s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.753510229 (425h18m14.624593264s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.755262356 (425h18m14.626345391s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.756625842 (425h18m14.627708877s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.758025117 (425h18m14.629108172s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.760642110 (425h18m14.631725145s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.761429270 (425h18m14.632512315s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763241921 (425h18m14.634324966s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763973835 (425h18m14.63505688s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765312685 (425h18m14.63639573s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765648824 (425h18m14.636731869s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.767253601 (425h18m14.638336636s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.768200434 (425h18m14.639283469s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:16.245896991 (425h18m16.116980036s) [upload=373358]: Next returned nil segment Dec 23 08:47:16.245944181 (425h18m16.117027226s) [upload=373358]: Waiting for error group managing segments... Dec 23 08:47:16.644138110 (425h18m16.515221145s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.131664201 (425h18m17.002747236s) [noise=true , address=64.49.13.139:4475 , piece_id=TH3UWCMUNGGADRA26PZNKZLOWCLRLRWIT23ONF2B2DJ3NMMFPZBQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.444602902 (425h18m17.315685947s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.470421138 (425h18m17.341504173s) [noise=true , address=77.90.31.13:4630 , piece_id=SOJYUZH676PZYM4Z5YGEXW4ZXPAVLSEIFFZBE22LGAUOMRRKVG7A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.554527268 (425h18m17.425610313s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.059191355 (425h18m17.93027439s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.354549959 (425h18m18.225633004s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.436262252 (425h18m18.307345297s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.800612455 (425h18m18.67169549s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.973704980 (425h18m18.844788025s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.276001403 (425h18m19.147084438s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.339083580 (425h18m19.210166625s) [noise=true , address=77.90.31.19:4876 , piece_id=EO3PO3DW5CXWWO4T6ND7J4JTZYC3RI35MLMUEQXQVEABP63LQMQQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.349762695 (425h18m19.22084573s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636789835 (425h18m20.50787288s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636817878 (425h18m20.507900923s) [seg_pos={0 0}, upload=373358]: Segment reached optimal threshold of 14 pieces. Dec 23 08:47:20.637067674 (425h18m20.508150709s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf): protocol: expected piece hash; context canceled; EOF Dec 23 08:47:20.637077883 (425h18m20.508160938s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637240834 (425h18m20.508323879s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu): protocol: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection Dec 23 08:47:20.637249079 (425h18m20.508332125s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637346505 (425h18m20.50842956s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bFEmP78qGhyHhXr3QNVptdiWQ4No4nrb31qj6DPgmJLcHYKiM): protocol: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection Dec 23 08:47:20.637368196 (425h18m20.508451241s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637562967 (425h18m20.508646012s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1acQG8MjLvgzmCs8ELswcsKJrNvYmGQLX1FL4dHeCj2AUH7A5R): protocol: read tcp 188.166.225.170:56478->45.91.116.13:4170: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 23 08:47:20.637572605 (425h18m20.50865564s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637715547 (425h18m20.508798592s) [seg_pos={0 0}, upload=373358]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 23 08:47:20.637871194 (425h18m20.508954239s) [upload=373358]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 08:47:20.637878588 (425h18m20.508961633s) [upload=373358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 08:47:21.234004555 (425h18m21.10508763s) [upload=373358]: Done uploading segments... Dec 23 08:47:21.234072815 (425h18m21.10515586s) [upload=373358]: Upload finished. err: <nil> Duration: 18.062834215s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 1.926151595s trace: https://jaeger.dev.storj.io/trace/3727d7c571aed99f FAIL (2025-12-23 09:08:47 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.926180721s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 1.420675369s trace: https://jaeger.dev.storj.io/trace/292b9465c989993a FAIL (2025-12-23 09:08:45 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 1.420707661s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 2.02935878s trace: https://jaeger.dev.storj.io/trace/3727d7c571aed99f FAIL (2025-12-23 09:08:32 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.029557278s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 6.947199115s trace: https://jaeger.dev.storj.io/trace/fbd33bd94283fbe FAIL (2025-12-23 09:08:31 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 6.947228299s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.02289532s trace: https://jaeger.dev.storj.io/trace/3c39be5f434273aa FAIL (2025-12-23 09:08:24 UTC) orbiter warning: context deadline exceeded Duration: 6.022921039s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.064071436s trace: https://jaeger.dev.storj.io/trace/265f3259a7705b6d FAIL (2025-12-23 09:08:23 UTC) orbiter warning: operation: sync2.Copy uplink: metaclient: context deadline exceeded Duration: 5.064216782s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 7.61385621s trace: https://jaeger.dev.storj.io/trace/6156a7d944a046d FAIL (2025-12-23 09:08:07 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 7.613895885s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 21.143460307s trace: https://jaeger.dev.storj.io/trace/3b621a03316dcb85 FAIL (2025-12-23 09:08:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1NgPjVCyZZAfxVr3WvKrGRjnkpPtUvmQcCMiYntxNKVSWqXcHC, Piece ID: MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ) context deadline exceeded; piecestore: (Node ID: 14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj, Piece ID: SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ) context deadline exceeded; piecestore: (Node ID: 124taxpXdSnzEkq6UG1ZPi7cKLkaoDSc3G9FH1YHbGbDwMTM8PG, Piece ID: XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA) context deadline exceeded; piecestore: (Node ID: 121nTAu2qwnGorPfRFRJ7kE4hSsPq9p8zCZVFtt7cNNGu1HGdnN, Piece ID: KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA) context deadline exceeded ======LOGS====== 3358]: Uploading piece... Dec 23 08:47:14.746900773 (425h18m14.617983818s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.749939840 (425h18m14.621022885s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.753510229 (425h18m14.624593264s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.755262356 (425h18m14.626345391s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.756625842 (425h18m14.627708877s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.758025117 (425h18m14.629108172s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.760642110 (425h18m14.631725145s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.761429270 (425h18m14.632512315s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763241921 (425h18m14.634324966s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763973835 (425h18m14.63505688s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765312685 (425h18m14.63639573s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765648824 (425h18m14.636731869s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.767253601 (425h18m14.638336636s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.768200434 (425h18m14.639283469s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:16.245896991 (425h18m16.116980036s) [upload=373358]: Next returned nil segment Dec 23 08:47:16.245944181 (425h18m16.117027226s) [upload=373358]: Waiting for error group managing segments... Dec 23 08:47:16.644138110 (425h18m16.515221145s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.131664201 (425h18m17.002747236s) [noise=true , address=64.49.13.139:4475 , piece_id=TH3UWCMUNGGADRA26PZNKZLOWCLRLRWIT23ONF2B2DJ3NMMFPZBQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.444602902 (425h18m17.315685947s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.470421138 (425h18m17.341504173s) [noise=true , address=77.90.31.13:4630 , piece_id=SOJYUZH676PZYM4Z5YGEXW4ZXPAVLSEIFFZBE22LGAUOMRRKVG7A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.554527268 (425h18m17.425610313s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.059191355 (425h18m17.93027439s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.354549959 (425h18m18.225633004s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.436262252 (425h18m18.307345297s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.800612455 (425h18m18.67169549s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.973704980 (425h18m18.844788025s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.276001403 (425h18m19.147084438s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.339083580 (425h18m19.210166625s) [noise=true , address=77.90.31.19:4876 , piece_id=EO3PO3DW5CXWWO4T6ND7J4JTZYC3RI35MLMUEQXQVEABP63LQMQQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.349762695 (425h18m19.22084573s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636789835 (425h18m20.50787288s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636817878 (425h18m20.507900923s) [seg_pos={0 0}, upload=373358]: Segment reached optimal threshold of 14 pieces. Dec 23 08:47:20.637067674 (425h18m20.508150709s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf): protocol: expected piece hash; context canceled; EOF Dec 23 08:47:20.637077883 (425h18m20.508160938s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637240834 (425h18m20.508323879s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu): protocol: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection Dec 23 08:47:20.637249079 (425h18m20.508332125s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637346505 (425h18m20.50842956s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bFEmP78qGhyHhXr3QNVptdiWQ4No4nrb31qj6DPgmJLcHYKiM): protocol: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection Dec 23 08:47:20.637368196 (425h18m20.508451241s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637562967 (425h18m20.508646012s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1acQG8MjLvgzmCs8ELswcsKJrNvYmGQLX1FL4dHeCj2AUH7A5R): protocol: read tcp 188.166.225.170:56478->45.91.116.13:4170: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 23 08:47:20.637572605 (425h18m20.50865564s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637715547 (425h18m20.508798592s) [seg_pos={0 0}, upload=373358]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 23 08:47:20.637871194 (425h18m20.508954239s) [upload=373358]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 08:47:20.637878588 (425h18m20.508961633s) [upload=373358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 08:47:21.234004555 (425h18m21.10508763s) [upload=373358]: Done uploading segments... Dec 23 08:47:21.234072815 (425h18m21.10515586s) [upload=373358]: Upload finished. err: <nil> Duration: 21.143484362s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.010061369s trace: https://jaeger.dev.storj.io/trace/265f3259a7705b6d FAIL (2025-12-23 09:07:55 UTC) orbiter warning: operation: sync2.Copy uplink: metaclient: context deadline exceeded Duration: 5.010095194s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 1.427191045s trace: https://jaeger.dev.storj.io/trace/292b9465c989993a FAIL (2025-12-23 09:07:38 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 1.427222316s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.838689074s trace: https://jaeger.dev.storj.io/trace/fbd33bd94283fbe FAIL (2025-12-23 09:07:37 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.838716697s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 2.017891846s trace: https://jaeger.dev.storj.io/trace/3727d7c571aed99f FAIL (2025-12-23 09:07:34 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.017923335s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.102586563s trace: https://jaeger.dev.storj.io/trace/265f3259a7705b6d FAIL (2025-12-23 09:07:33 UTC) orbiter warning: operation: sync2.Copy uplink: metaclient: context deadline exceeded Duration: 5.102616421s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 10.360330276s trace: https://jaeger.dev.storj.io/trace/3b621a03316dcb85 FAIL (2025-12-23 09:07:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1NgPjVCyZZAfxVr3WvKrGRjnkpPtUvmQcCMiYntxNKVSWqXcHC, Piece ID: MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ) context deadline exceeded; piecestore: (Node ID: 14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj, Piece ID: SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ) context deadline exceeded; piecestore: (Node ID: 124taxpXdSnzEkq6UG1ZPi7cKLkaoDSc3G9FH1YHbGbDwMTM8PG, Piece ID: XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA) context deadline exceeded; piecestore: (Node ID: 121nTAu2qwnGorPfRFRJ7kE4hSsPq9p8zCZVFtt7cNNGu1HGdnN, Piece ID: KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA) context deadline exceeded ======LOGS====== 3358]: Uploading piece... Dec 23 08:47:14.746900773 (425h18m14.617983818s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.749939840 (425h18m14.621022885s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.753510229 (425h18m14.624593264s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.755262356 (425h18m14.626345391s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.756625842 (425h18m14.627708877s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.758025117 (425h18m14.629108172s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.760642110 (425h18m14.631725145s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.761429270 (425h18m14.632512315s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763241921 (425h18m14.634324966s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763973835 (425h18m14.63505688s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765312685 (425h18m14.63639573s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765648824 (425h18m14.636731869s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.767253601 (425h18m14.638336636s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.768200434 (425h18m14.639283469s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:16.245896991 (425h18m16.116980036s) [upload=373358]: Next returned nil segment Dec 23 08:47:16.245944181 (425h18m16.117027226s) [upload=373358]: Waiting for error group managing segments... Dec 23 08:47:16.644138110 (425h18m16.515221145s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.131664201 (425h18m17.002747236s) [noise=true , address=64.49.13.139:4475 , piece_id=TH3UWCMUNGGADRA26PZNKZLOWCLRLRWIT23ONF2B2DJ3NMMFPZBQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.444602902 (425h18m17.315685947s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.470421138 (425h18m17.341504173s) [noise=true , address=77.90.31.13:4630 , piece_id=SOJYUZH676PZYM4Z5YGEXW4ZXPAVLSEIFFZBE22LGAUOMRRKVG7A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.554527268 (425h18m17.425610313s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.059191355 (425h18m17.93027439s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.354549959 (425h18m18.225633004s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.436262252 (425h18m18.307345297s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.800612455 (425h18m18.67169549s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.973704980 (425h18m18.844788025s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.276001403 (425h18m19.147084438s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.339083580 (425h18m19.210166625s) [noise=true , address=77.90.31.19:4876 , piece_id=EO3PO3DW5CXWWO4T6ND7J4JTZYC3RI35MLMUEQXQVEABP63LQMQQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.349762695 (425h18m19.22084573s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636789835 (425h18m20.50787288s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636817878 (425h18m20.507900923s) [seg_pos={0 0}, upload=373358]: Segment reached optimal threshold of 14 pieces. Dec 23 08:47:20.637067674 (425h18m20.508150709s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf): protocol: expected piece hash; context canceled; EOF Dec 23 08:47:20.637077883 (425h18m20.508160938s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637240834 (425h18m20.508323879s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu): protocol: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection Dec 23 08:47:20.637249079 (425h18m20.508332125s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637346505 (425h18m20.50842956s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bFEmP78qGhyHhXr3QNVptdiWQ4No4nrb31qj6DPgmJLcHYKiM): protocol: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection Dec 23 08:47:20.637368196 (425h18m20.508451241s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637562967 (425h18m20.508646012s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1acQG8MjLvgzmCs8ELswcsKJrNvYmGQLX1FL4dHeCj2AUH7A5R): protocol: read tcp 188.166.225.170:56478->45.91.116.13:4170: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 23 08:47:20.637572605 (425h18m20.50865564s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637715547 (425h18m20.508798592s) [seg_pos={0 0}, upload=373358]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 23 08:47:20.637871194 (425h18m20.508954239s) [upload=373358]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 08:47:20.637878588 (425h18m20.508961633s) [upload=373358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 08:47:21.234004555 (425h18m21.10508763s) [upload=373358]: Done uploading segments... Dec 23 08:47:21.234072815 (425h18m21.10515586s) [upload=373358]: Upload finished. err: <nil> Duration: 10.360357668s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.655536259s trace: https://jaeger.dev.storj.io/trace/3c39be5f434273aa FAIL (2025-12-23 09:07:27 UTC) orbiter warning: context deadline exceeded Duration: 5.655566136s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.728912859s trace: https://jaeger.dev.storj.io/trace/6156a7d944a046d FAIL (2025-12-23 09:07:19 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 6.728946905s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.451324652s trace: https://jaeger.dev.storj.io/trace/fbd33bd94283fbe FAIL (2025-12-23 09:07:09 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 6.451354277s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 1.940580421s trace: https://jaeger.dev.storj.io/trace/3727d7c571aed99f FAIL (2025-12-23 09:07:08 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.940609526s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 1.421959363s trace: https://jaeger.dev.storj.io/trace/292b9465c989993a FAIL (2025-12-23 09:07:05 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 1.421992026s core uplink eu1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.354731882s trace: https://jaeger.dev.storj.io/trace/3c39be5f434273aa FAIL (2025-12-23 09:06:56 UTC) orbiter warning: context deadline exceeded Duration: 6.35475176s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 14.220077112s trace: https://jaeger.dev.storj.io/trace/3b621a03316dcb85 FAIL (2025-12-23 09:06:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1NgPjVCyZZAfxVr3WvKrGRjnkpPtUvmQcCMiYntxNKVSWqXcHC, Piece ID: MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ) context deadline exceeded; piecestore: (Node ID: 14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj, Piece ID: SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ) context deadline exceeded; piecestore: (Node ID: 124taxpXdSnzEkq6UG1ZPi7cKLkaoDSc3G9FH1YHbGbDwMTM8PG, Piece ID: XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA) context deadline exceeded; piecestore: (Node ID: 121nTAu2qwnGorPfRFRJ7kE4hSsPq9p8zCZVFtt7cNNGu1HGdnN, Piece ID: KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA) context deadline exceeded ======LOGS====== 3358]: Uploading piece... Dec 23 08:47:14.746900773 (425h18m14.617983818s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.749939840 (425h18m14.621022885s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.753510229 (425h18m14.624593264s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.755262356 (425h18m14.626345391s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.756625842 (425h18m14.627708877s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.758025117 (425h18m14.629108172s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.760642110 (425h18m14.631725145s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.761429270 (425h18m14.632512315s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763241921 (425h18m14.634324966s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763973835 (425h18m14.63505688s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765312685 (425h18m14.63639573s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765648824 (425h18m14.636731869s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.767253601 (425h18m14.638336636s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.768200434 (425h18m14.639283469s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:16.245896991 (425h18m16.116980036s) [upload=373358]: Next returned nil segment Dec 23 08:47:16.245944181 (425h18m16.117027226s) [upload=373358]: Waiting for error group managing segments... Dec 23 08:47:16.644138110 (425h18m16.515221145s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.131664201 (425h18m17.002747236s) [noise=true , address=64.49.13.139:4475 , piece_id=TH3UWCMUNGGADRA26PZNKZLOWCLRLRWIT23ONF2B2DJ3NMMFPZBQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.444602902 (425h18m17.315685947s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.470421138 (425h18m17.341504173s) [noise=true , address=77.90.31.13:4630 , piece_id=SOJYUZH676PZYM4Z5YGEXW4ZXPAVLSEIFFZBE22LGAUOMRRKVG7A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.554527268 (425h18m17.425610313s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.059191355 (425h18m17.93027439s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.354549959 (425h18m18.225633004s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.436262252 (425h18m18.307345297s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.800612455 (425h18m18.67169549s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.973704980 (425h18m18.844788025s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.276001403 (425h18m19.147084438s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.339083580 (425h18m19.210166625s) [noise=true , address=77.90.31.19:4876 , piece_id=EO3PO3DW5CXWWO4T6ND7J4JTZYC3RI35MLMUEQXQVEABP63LQMQQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.349762695 (425h18m19.22084573s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636789835 (425h18m20.50787288s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636817878 (425h18m20.507900923s) [seg_pos={0 0}, upload=373358]: Segment reached optimal threshold of 14 pieces. Dec 23 08:47:20.637067674 (425h18m20.508150709s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf): protocol: expected piece hash; context canceled; EOF Dec 23 08:47:20.637077883 (425h18m20.508160938s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637240834 (425h18m20.508323879s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu): protocol: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection Dec 23 08:47:20.637249079 (425h18m20.508332125s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637346505 (425h18m20.50842956s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bFEmP78qGhyHhXr3QNVptdiWQ4No4nrb31qj6DPgmJLcHYKiM): protocol: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection Dec 23 08:47:20.637368196 (425h18m20.508451241s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637562967 (425h18m20.508646012s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1acQG8MjLvgzmCs8ELswcsKJrNvYmGQLX1FL4dHeCj2AUH7A5R): protocol: read tcp 188.166.225.170:56478->45.91.116.13:4170: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 23 08:47:20.637572605 (425h18m20.50865564s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637715547 (425h18m20.508798592s) [seg_pos={0 0}, upload=373358]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 23 08:47:20.637871194 (425h18m20.508954239s) [upload=373358]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 08:47:20.637878588 (425h18m20.508961633s) [upload=373358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 08:47:21.234004555 (425h18m21.10508763s) [upload=373358]: Done uploading segments... Dec 23 08:47:21.234072815 (425h18m21.10515586s) [upload=373358]: Upload finished. err: <nil> Duration: 14.220104695s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.83393412s trace: https://jaeger.dev.storj.io/trace/fbd33bd94283fbe FAIL (2025-12-23 09:06:50 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.833959358s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.185461432s trace: https://jaeger.dev.storj.io/trace/265f3259a7705b6d FAIL (2025-12-23 09:06:47 UTC) orbiter warning: operation: sync2.Copy uplink: metaclient: context deadline exceeded Duration: 5.185488993s core uplink eu1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.37259163s trace: https://jaeger.dev.storj.io/trace/3c39be5f434273aa FAIL (2025-12-23 09:06:40 UTC) orbiter warning: context deadline exceeded Duration: 6.37261745s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 7.490822891s trace: https://jaeger.dev.storj.io/trace/6156a7d944a046d FAIL (2025-12-23 09:06:31 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 7.490855985s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.90791336s trace: https://jaeger.dev.storj.io/trace/3c39be5f434273aa FAIL (2025-12-23 09:06:16 UTC) orbiter warning: context deadline exceeded Duration: 5.907946072s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 16.338643929s trace: https://jaeger.dev.storj.io/trace/3b621a03316dcb85 FAIL (2025-12-23 09:06:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1NgPjVCyZZAfxVr3WvKrGRjnkpPtUvmQcCMiYntxNKVSWqXcHC, Piece ID: MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ) context deadline exceeded; piecestore: (Node ID: 14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj, Piece ID: SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ) context deadline exceeded; piecestore: (Node ID: 124taxpXdSnzEkq6UG1ZPi7cKLkaoDSc3G9FH1YHbGbDwMTM8PG, Piece ID: XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA) context deadline exceeded; piecestore: (Node ID: 121nTAu2qwnGorPfRFRJ7kE4hSsPq9p8zCZVFtt7cNNGu1HGdnN, Piece ID: KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA) context deadline exceeded ======LOGS====== 3358]: Uploading piece... Dec 23 08:47:14.746900773 (425h18m14.617983818s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.749939840 (425h18m14.621022885s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.753510229 (425h18m14.624593264s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.755262356 (425h18m14.626345391s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.756625842 (425h18m14.627708877s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.758025117 (425h18m14.629108172s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.760642110 (425h18m14.631725145s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.761429270 (425h18m14.632512315s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763241921 (425h18m14.634324966s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763973835 (425h18m14.63505688s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765312685 (425h18m14.63639573s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765648824 (425h18m14.636731869s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.767253601 (425h18m14.638336636s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.768200434 (425h18m14.639283469s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:16.245896991 (425h18m16.116980036s) [upload=373358]: Next returned nil segment Dec 23 08:47:16.245944181 (425h18m16.117027226s) [upload=373358]: Waiting for error group managing segments... Dec 23 08:47:16.644138110 (425h18m16.515221145s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.131664201 (425h18m17.002747236s) [noise=true , address=64.49.13.139:4475 , piece_id=TH3UWCMUNGGADRA26PZNKZLOWCLRLRWIT23ONF2B2DJ3NMMFPZBQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.444602902 (425h18m17.315685947s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.470421138 (425h18m17.341504173s) [noise=true , address=77.90.31.13:4630 , piece_id=SOJYUZH676PZYM4Z5YGEXW4ZXPAVLSEIFFZBE22LGAUOMRRKVG7A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.554527268 (425h18m17.425610313s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.059191355 (425h18m17.93027439s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.354549959 (425h18m18.225633004s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.436262252 (425h18m18.307345297s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.800612455 (425h18m18.67169549s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.973704980 (425h18m18.844788025s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.276001403 (425h18m19.147084438s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.339083580 (425h18m19.210166625s) [noise=true , address=77.90.31.19:4876 , piece_id=EO3PO3DW5CXWWO4T6ND7J4JTZYC3RI35MLMUEQXQVEABP63LQMQQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.349762695 (425h18m19.22084573s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636789835 (425h18m20.50787288s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636817878 (425h18m20.507900923s) [seg_pos={0 0}, upload=373358]: Segment reached optimal threshold of 14 pieces. Dec 23 08:47:20.637067674 (425h18m20.508150709s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf): protocol: expected piece hash; context canceled; EOF Dec 23 08:47:20.637077883 (425h18m20.508160938s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637240834 (425h18m20.508323879s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu): protocol: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection Dec 23 08:47:20.637249079 (425h18m20.508332125s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637346505 (425h18m20.50842956s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bFEmP78qGhyHhXr3QNVptdiWQ4No4nrb31qj6DPgmJLcHYKiM): protocol: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection Dec 23 08:47:20.637368196 (425h18m20.508451241s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637562967 (425h18m20.508646012s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1acQG8MjLvgzmCs8ELswcsKJrNvYmGQLX1FL4dHeCj2AUH7A5R): protocol: read tcp 188.166.225.170:56478->45.91.116.13:4170: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 23 08:47:20.637572605 (425h18m20.50865564s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637715547 (425h18m20.508798592s) [seg_pos={0 0}, upload=373358]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 23 08:47:20.637871194 (425h18m20.508954239s) [upload=373358]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 08:47:20.637878588 (425h18m20.508961633s) [upload=373358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 08:47:21.234004555 (425h18m21.10508763s) [upload=373358]: Done uploading segments... Dec 23 08:47:21.234072815 (425h18m21.10515586s) [upload=373358]: Upload finished. err: <nil> Duration: 16.338672603s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 2.68814168s trace: https://jaeger.dev.storj.io/trace/3727d7c571aed99f FAIL (2025-12-23 09:06:04 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.688192698s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 1.460994625s trace: https://jaeger.dev.storj.io/trace/292b9465c989993a FAIL (2025-12-23 09:06:04 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 1.461020204s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 8.101417355s trace: https://jaeger.dev.storj.io/trace/6156a7d944a046d FAIL (2025-12-23 09:05:54 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 8.101434298s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)