Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 7.6110896s trace: https://jaeger.dev.storj.io/trace/6156a7d944a046d FAIL (2025-12-23 09:14:53 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 7.611117494s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.03820514s trace: https://jaeger.dev.storj.io/trace/265f3259a7705b6d FAIL (2025-12-23 09:14:48 UTC) orbiter warning: operation: sync2.Copy uplink: metaclient: context deadline exceeded Duration: 5.038237923s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 15.219774484s trace: https://jaeger.dev.storj.io/trace/3b621a03316dcb85 FAIL (2025-12-23 09:14:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1NgPjVCyZZAfxVr3WvKrGRjnkpPtUvmQcCMiYntxNKVSWqXcHC, Piece ID: MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ) context deadline exceeded; piecestore: (Node ID: 14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj, Piece ID: SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ) context deadline exceeded; piecestore: (Node ID: 124taxpXdSnzEkq6UG1ZPi7cKLkaoDSc3G9FH1YHbGbDwMTM8PG, Piece ID: XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA) context deadline exceeded; piecestore: (Node ID: 121nTAu2qwnGorPfRFRJ7kE4hSsPq9p8zCZVFtt7cNNGu1HGdnN, Piece ID: KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA) context deadline exceeded ======LOGS====== 3358]: Uploading piece... Dec 23 08:47:14.746900773 (425h18m14.617983818s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.749939840 (425h18m14.621022885s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.753510229 (425h18m14.624593264s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.755262356 (425h18m14.626345391s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.756625842 (425h18m14.627708877s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.758025117 (425h18m14.629108172s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.760642110 (425h18m14.631725145s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.761429270 (425h18m14.632512315s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763241921 (425h18m14.634324966s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763973835 (425h18m14.63505688s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765312685 (425h18m14.63639573s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765648824 (425h18m14.636731869s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.767253601 (425h18m14.638336636s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.768200434 (425h18m14.639283469s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:16.245896991 (425h18m16.116980036s) [upload=373358]: Next returned nil segment Dec 23 08:47:16.245944181 (425h18m16.117027226s) [upload=373358]: Waiting for error group managing segments... Dec 23 08:47:16.644138110 (425h18m16.515221145s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.131664201 (425h18m17.002747236s) [noise=true , address=64.49.13.139:4475 , piece_id=TH3UWCMUNGGADRA26PZNKZLOWCLRLRWIT23ONF2B2DJ3NMMFPZBQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.444602902 (425h18m17.315685947s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.470421138 (425h18m17.341504173s) [noise=true , address=77.90.31.13:4630 , piece_id=SOJYUZH676PZYM4Z5YGEXW4ZXPAVLSEIFFZBE22LGAUOMRRKVG7A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.554527268 (425h18m17.425610313s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.059191355 (425h18m17.93027439s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.354549959 (425h18m18.225633004s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.436262252 (425h18m18.307345297s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.800612455 (425h18m18.67169549s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.973704980 (425h18m18.844788025s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.276001403 (425h18m19.147084438s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.339083580 (425h18m19.210166625s) [noise=true , address=77.90.31.19:4876 , piece_id=EO3PO3DW5CXWWO4T6ND7J4JTZYC3RI35MLMUEQXQVEABP63LQMQQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.349762695 (425h18m19.22084573s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636789835 (425h18m20.50787288s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636817878 (425h18m20.507900923s) [seg_pos={0 0}, upload=373358]: Segment reached optimal threshold of 14 pieces. Dec 23 08:47:20.637067674 (425h18m20.508150709s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf): protocol: expected piece hash; context canceled; EOF Dec 23 08:47:20.637077883 (425h18m20.508160938s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637240834 (425h18m20.508323879s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu): protocol: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection Dec 23 08:47:20.637249079 (425h18m20.508332125s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637346505 (425h18m20.50842956s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bFEmP78qGhyHhXr3QNVptdiWQ4No4nrb31qj6DPgmJLcHYKiM): protocol: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection Dec 23 08:47:20.637368196 (425h18m20.508451241s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637562967 (425h18m20.508646012s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1acQG8MjLvgzmCs8ELswcsKJrNvYmGQLX1FL4dHeCj2AUH7A5R): protocol: read tcp 188.166.225.170:56478->45.91.116.13:4170: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 23 08:47:20.637572605 (425h18m20.50865564s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637715547 (425h18m20.508798592s) [seg_pos={0 0}, upload=373358]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 23 08:47:20.637871194 (425h18m20.508954239s) [upload=373358]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 08:47:20.637878588 (425h18m20.508961633s) [upload=373358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 08:47:21.234004555 (425h18m21.10508763s) [upload=373358]: Done uploading segments... Dec 23 08:47:21.234072815 (425h18m21.10515586s) [upload=373358]: Upload finished. err: <nil> Duration: 15.219805112s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.446557496s trace: https://jaeger.dev.storj.io/trace/265f3259a7705b6d FAIL (2025-12-23 09:14:22 UTC) orbiter warning: operation: sync2.Copy uplink: metaclient: context deadline exceeded Duration: 5.446587876s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 1.379551231s trace: https://jaeger.dev.storj.io/trace/292b9465c989993a FAIL (2025-12-23 09:14:20 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 1.379582461s core uplink eu1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.639811936s trace: https://jaeger.dev.storj.io/trace/3c39be5f434273aa FAIL (2025-12-23 09:14:19 UTC) orbiter warning: context deadline exceeded Duration: 5.639836714s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 9.872038241s trace: https://jaeger.dev.storj.io/trace/6156a7d944a046d FAIL (2025-12-23 09:14:16 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 9.872067747s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.544103827s trace: https://jaeger.dev.storj.io/trace/3727d7c571aed99f FAIL (2025-12-23 09:14:13 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.544245516s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 7.516257925s trace: https://jaeger.dev.storj.io/trace/fbd33bd94283fbe FAIL (2025-12-23 09:14:12 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 7.516287921s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.232443496s trace: https://jaeger.dev.storj.io/trace/265f3259a7705b6d FAIL (2025-12-23 09:14:02 UTC) orbiter warning: operation: sync2.Copy uplink: metaclient: context deadline exceeded Duration: 5.232467512s core uplink eu1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.274594888s trace: https://jaeger.dev.storj.io/trace/3c39be5f434273aa FAIL (2025-12-23 09:13:58 UTC) orbiter warning: context deadline exceeded Duration: 6.274617962s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 18.667972878s trace: https://jaeger.dev.storj.io/trace/3b621a03316dcb85 FAIL (2025-12-23 09:13:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1NgPjVCyZZAfxVr3WvKrGRjnkpPtUvmQcCMiYntxNKVSWqXcHC, Piece ID: MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ) context deadline exceeded; piecestore: (Node ID: 14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj, Piece ID: SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ) context deadline exceeded; piecestore: (Node ID: 124taxpXdSnzEkq6UG1ZPi7cKLkaoDSc3G9FH1YHbGbDwMTM8PG, Piece ID: XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA) context deadline exceeded; piecestore: (Node ID: 121nTAu2qwnGorPfRFRJ7kE4hSsPq9p8zCZVFtt7cNNGu1HGdnN, Piece ID: KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA) context deadline exceeded ======LOGS====== 3358]: Uploading piece... Dec 23 08:47:14.746900773 (425h18m14.617983818s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.749939840 (425h18m14.621022885s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.753510229 (425h18m14.624593264s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.755262356 (425h18m14.626345391s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.756625842 (425h18m14.627708877s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.758025117 (425h18m14.629108172s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.760642110 (425h18m14.631725145s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.761429270 (425h18m14.632512315s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763241921 (425h18m14.634324966s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763973835 (425h18m14.63505688s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765312685 (425h18m14.63639573s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765648824 (425h18m14.636731869s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.767253601 (425h18m14.638336636s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.768200434 (425h18m14.639283469s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:16.245896991 (425h18m16.116980036s) [upload=373358]: Next returned nil segment Dec 23 08:47:16.245944181 (425h18m16.117027226s) [upload=373358]: Waiting for error group managing segments... Dec 23 08:47:16.644138110 (425h18m16.515221145s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.131664201 (425h18m17.002747236s) [noise=true , address=64.49.13.139:4475 , piece_id=TH3UWCMUNGGADRA26PZNKZLOWCLRLRWIT23ONF2B2DJ3NMMFPZBQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.444602902 (425h18m17.315685947s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.470421138 (425h18m17.341504173s) [noise=true , address=77.90.31.13:4630 , piece_id=SOJYUZH676PZYM4Z5YGEXW4ZXPAVLSEIFFZBE22LGAUOMRRKVG7A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.554527268 (425h18m17.425610313s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.059191355 (425h18m17.93027439s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.354549959 (425h18m18.225633004s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.436262252 (425h18m18.307345297s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.800612455 (425h18m18.67169549s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.973704980 (425h18m18.844788025s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.276001403 (425h18m19.147084438s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.339083580 (425h18m19.210166625s) [noise=true , address=77.90.31.19:4876 , piece_id=EO3PO3DW5CXWWO4T6ND7J4JTZYC3RI35MLMUEQXQVEABP63LQMQQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.349762695 (425h18m19.22084573s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636789835 (425h18m20.50787288s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636817878 (425h18m20.507900923s) [seg_pos={0 0}, upload=373358]: Segment reached optimal threshold of 14 pieces. Dec 23 08:47:20.637067674 (425h18m20.508150709s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf): protocol: expected piece hash; context canceled; EOF Dec 23 08:47:20.637077883 (425h18m20.508160938s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637240834 (425h18m20.508323879s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu): protocol: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection Dec 23 08:47:20.637249079 (425h18m20.508332125s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637346505 (425h18m20.50842956s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bFEmP78qGhyHhXr3QNVptdiWQ4No4nrb31qj6DPgmJLcHYKiM): protocol: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection Dec 23 08:47:20.637368196 (425h18m20.508451241s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637562967 (425h18m20.508646012s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1acQG8MjLvgzmCs8ELswcsKJrNvYmGQLX1FL4dHeCj2AUH7A5R): protocol: read tcp 188.166.225.170:56478->45.91.116.13:4170: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 23 08:47:20.637572605 (425h18m20.50865564s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637715547 (425h18m20.508798592s) [seg_pos={0 0}, upload=373358]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 23 08:47:20.637871194 (425h18m20.508954239s) [upload=373358]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 08:47:20.637878588 (425h18m20.508961633s) [upload=373358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 08:47:21.234004555 (425h18m21.10508763s) [upload=373358]: Done uploading segments... Dec 23 08:47:21.234072815 (425h18m21.10515586s) [upload=373358]: Upload finished. err: <nil> Duration: 18.668001141s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.198153562s trace: https://jaeger.dev.storj.io/trace/265f3259a7705b6d FAIL (2025-12-23 09:13:43 UTC) orbiter warning: operation: sync2.Copy uplink: metaclient: context deadline exceeded Duration: 5.198182836s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 1.442724244s trace: https://jaeger.dev.storj.io/trace/292b9465c989993a FAIL (2025-12-23 09:13:41 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 1.442750885s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 1.95626103s trace: https://jaeger.dev.storj.io/trace/3727d7c571aed99f FAIL (2025-12-23 09:13:40 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.95628718s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 7.031157748s trace: https://jaeger.dev.storj.io/trace/6156a7d944a046d FAIL (2025-12-23 09:13:27 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 7.031182014s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 6.572419879s trace: https://jaeger.dev.storj.io/trace/fbd33bd94283fbe FAIL (2025-12-23 09:13:26 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 6.572449004s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.682235358s trace: https://jaeger.dev.storj.io/trace/3c39be5f434273aa FAIL (2025-12-23 09:12:52 UTC) orbiter warning: context deadline exceeded Duration: 6.682267059s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 25.656322393s trace: https://jaeger.dev.storj.io/trace/3b621a03316dcb85 FAIL (2025-12-23 09:12:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1NgPjVCyZZAfxVr3WvKrGRjnkpPtUvmQcCMiYntxNKVSWqXcHC, Piece ID: MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ) context deadline exceeded; piecestore: (Node ID: 14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj, Piece ID: SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ) context deadline exceeded; piecestore: (Node ID: 124taxpXdSnzEkq6UG1ZPi7cKLkaoDSc3G9FH1YHbGbDwMTM8PG, Piece ID: XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA) context deadline exceeded; piecestore: (Node ID: 121nTAu2qwnGorPfRFRJ7kE4hSsPq9p8zCZVFtt7cNNGu1HGdnN, Piece ID: KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA) context deadline exceeded ======LOGS====== 3358]: Uploading piece... Dec 23 08:47:14.746900773 (425h18m14.617983818s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.749939840 (425h18m14.621022885s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.753510229 (425h18m14.624593264s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.755262356 (425h18m14.626345391s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.756625842 (425h18m14.627708877s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.758025117 (425h18m14.629108172s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.760642110 (425h18m14.631725145s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.761429270 (425h18m14.632512315s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763241921 (425h18m14.634324966s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763973835 (425h18m14.63505688s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765312685 (425h18m14.63639573s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765648824 (425h18m14.636731869s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.767253601 (425h18m14.638336636s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.768200434 (425h18m14.639283469s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:16.245896991 (425h18m16.116980036s) [upload=373358]: Next returned nil segment Dec 23 08:47:16.245944181 (425h18m16.117027226s) [upload=373358]: Waiting for error group managing segments... Dec 23 08:47:16.644138110 (425h18m16.515221145s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.131664201 (425h18m17.002747236s) [noise=true , address=64.49.13.139:4475 , piece_id=TH3UWCMUNGGADRA26PZNKZLOWCLRLRWIT23ONF2B2DJ3NMMFPZBQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.444602902 (425h18m17.315685947s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.470421138 (425h18m17.341504173s) [noise=true , address=77.90.31.13:4630 , piece_id=SOJYUZH676PZYM4Z5YGEXW4ZXPAVLSEIFFZBE22LGAUOMRRKVG7A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.554527268 (425h18m17.425610313s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.059191355 (425h18m17.93027439s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.354549959 (425h18m18.225633004s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.436262252 (425h18m18.307345297s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.800612455 (425h18m18.67169549s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.973704980 (425h18m18.844788025s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.276001403 (425h18m19.147084438s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.339083580 (425h18m19.210166625s) [noise=true , address=77.90.31.19:4876 , piece_id=EO3PO3DW5CXWWO4T6ND7J4JTZYC3RI35MLMUEQXQVEABP63LQMQQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.349762695 (425h18m19.22084573s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636789835 (425h18m20.50787288s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636817878 (425h18m20.507900923s) [seg_pos={0 0}, upload=373358]: Segment reached optimal threshold of 14 pieces. Dec 23 08:47:20.637067674 (425h18m20.508150709s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf): protocol: expected piece hash; context canceled; EOF Dec 23 08:47:20.637077883 (425h18m20.508160938s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637240834 (425h18m20.508323879s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu): protocol: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection Dec 23 08:47:20.637249079 (425h18m20.508332125s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637346505 (425h18m20.50842956s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bFEmP78qGhyHhXr3QNVptdiWQ4No4nrb31qj6DPgmJLcHYKiM): protocol: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection Dec 23 08:47:20.637368196 (425h18m20.508451241s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637562967 (425h18m20.508646012s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1acQG8MjLvgzmCs8ELswcsKJrNvYmGQLX1FL4dHeCj2AUH7A5R): protocol: read tcp 188.166.225.170:56478->45.91.116.13:4170: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 23 08:47:20.637572605 (425h18m20.50865564s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637715547 (425h18m20.508798592s) [seg_pos={0 0}, upload=373358]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 23 08:47:20.637871194 (425h18m20.508954239s) [upload=373358]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 08:47:20.637878588 (425h18m20.508961633s) [upload=373358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 08:47:21.234004555 (425h18m21.10508763s) [upload=373358]: Done uploading segments... Dec 23 08:47:21.234072815 (425h18m21.10515586s) [upload=373358]: Upload finished. err: <nil> Duration: 25.656349043s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.623271902s trace: https://jaeger.dev.storj.io/trace/265f3259a7705b6d FAIL (2025-12-23 09:12:47 UTC) orbiter warning: operation: sync2.Copy uplink: metaclient: context deadline exceeded Duration: 5.623296809s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 9.70780353s trace: https://jaeger.dev.storj.io/trace/6156a7d944a046d FAIL (2025-12-23 09:12:42 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 9.707837364s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 6.840367348s trace: https://jaeger.dev.storj.io/trace/fbd33bd94283fbe FAIL (2025-12-23 09:12:36 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 6.840385682s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 2.053205659s trace: https://jaeger.dev.storj.io/trace/3727d7c571aed99f FAIL (2025-12-23 09:12:33 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.053235095s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 1.417649316s trace: https://jaeger.dev.storj.io/trace/292b9465c989993a FAIL (2025-12-23 09:12:32 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 1.417677351s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.668937577s trace: https://jaeger.dev.storj.io/trace/265f3259a7705b6d FAIL (2025-12-23 09:12:16 UTC) orbiter warning: operation: sync2.Copy uplink: metaclient: context deadline exceeded Duration: 5.668964517s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 18.622562817s trace: https://jaeger.dev.storj.io/trace/3b621a03316dcb85 FAIL (2025-12-23 09:11:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1NgPjVCyZZAfxVr3WvKrGRjnkpPtUvmQcCMiYntxNKVSWqXcHC, Piece ID: MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ) context deadline exceeded; piecestore: (Node ID: 14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj, Piece ID: SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ) context deadline exceeded; piecestore: (Node ID: 124taxpXdSnzEkq6UG1ZPi7cKLkaoDSc3G9FH1YHbGbDwMTM8PG, Piece ID: XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA) context deadline exceeded; piecestore: (Node ID: 121nTAu2qwnGorPfRFRJ7kE4hSsPq9p8zCZVFtt7cNNGu1HGdnN, Piece ID: KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA) context deadline exceeded ======LOGS====== 3358]: Uploading piece... Dec 23 08:47:14.746900773 (425h18m14.617983818s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.749939840 (425h18m14.621022885s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.753510229 (425h18m14.624593264s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.755262356 (425h18m14.626345391s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.756625842 (425h18m14.627708877s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.758025117 (425h18m14.629108172s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.760642110 (425h18m14.631725145s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.761429270 (425h18m14.632512315s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763241921 (425h18m14.634324966s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763973835 (425h18m14.63505688s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765312685 (425h18m14.63639573s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765648824 (425h18m14.636731869s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.767253601 (425h18m14.638336636s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.768200434 (425h18m14.639283469s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:16.245896991 (425h18m16.116980036s) [upload=373358]: Next returned nil segment Dec 23 08:47:16.245944181 (425h18m16.117027226s) [upload=373358]: Waiting for error group managing segments... Dec 23 08:47:16.644138110 (425h18m16.515221145s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.131664201 (425h18m17.002747236s) [noise=true , address=64.49.13.139:4475 , piece_id=TH3UWCMUNGGADRA26PZNKZLOWCLRLRWIT23ONF2B2DJ3NMMFPZBQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.444602902 (425h18m17.315685947s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.470421138 (425h18m17.341504173s) [noise=true , address=77.90.31.13:4630 , piece_id=SOJYUZH676PZYM4Z5YGEXW4ZXPAVLSEIFFZBE22LGAUOMRRKVG7A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.554527268 (425h18m17.425610313s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.059191355 (425h18m17.93027439s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.354549959 (425h18m18.225633004s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.436262252 (425h18m18.307345297s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.800612455 (425h18m18.67169549s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.973704980 (425h18m18.844788025s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.276001403 (425h18m19.147084438s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.339083580 (425h18m19.210166625s) [noise=true , address=77.90.31.19:4876 , piece_id=EO3PO3DW5CXWWO4T6ND7J4JTZYC3RI35MLMUEQXQVEABP63LQMQQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.349762695 (425h18m19.22084573s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636789835 (425h18m20.50787288s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636817878 (425h18m20.507900923s) [seg_pos={0 0}, upload=373358]: Segment reached optimal threshold of 14 pieces. Dec 23 08:47:20.637067674 (425h18m20.508150709s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf): protocol: expected piece hash; context canceled; EOF Dec 23 08:47:20.637077883 (425h18m20.508160938s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637240834 (425h18m20.508323879s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu): protocol: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection Dec 23 08:47:20.637249079 (425h18m20.508332125s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637346505 (425h18m20.50842956s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bFEmP78qGhyHhXr3QNVptdiWQ4No4nrb31qj6DPgmJLcHYKiM): protocol: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection Dec 23 08:47:20.637368196 (425h18m20.508451241s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637562967 (425h18m20.508646012s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1acQG8MjLvgzmCs8ELswcsKJrNvYmGQLX1FL4dHeCj2AUH7A5R): protocol: read tcp 188.166.225.170:56478->45.91.116.13:4170: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 23 08:47:20.637572605 (425h18m20.50865564s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637715547 (425h18m20.508798592s) [seg_pos={0 0}, upload=373358]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 23 08:47:20.637871194 (425h18m20.508954239s) [upload=373358]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 08:47:20.637878588 (425h18m20.508961633s) [upload=373358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 08:47:21.234004555 (425h18m21.10508763s) [upload=373358]: Done uploading segments... Dec 23 08:47:21.234072815 (425h18m21.10515586s) [upload=373358]: Upload finished. err: <nil> Duration: 18.622587434s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 6.044000855s trace: https://jaeger.dev.storj.io/trace/fbd33bd94283fbe FAIL (2025-12-23 09:11:55 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 6.04403481s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 2.497620625s trace: https://jaeger.dev.storj.io/trace/3727d7c571aed99f FAIL (2025-12-23 09:11:40 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.497654789s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 7.50330003s trace: https://jaeger.dev.storj.io/trace/6156a7d944a046d FAIL (2025-12-23 09:11:39 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 7.50331611s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.97758138s trace: https://jaeger.dev.storj.io/trace/3c39be5f434273aa FAIL (2025-12-23 09:11:39 UTC) orbiter warning: context deadline exceeded Duration: 5.977609875s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 5.349201183s trace: https://jaeger.dev.storj.io/trace/265f3259a7705b6d FAIL (2025-12-23 09:11:32 UTC) orbiter warning: operation: sync2.Copy uplink: metaclient: context deadline exceeded Duration: 5.349236871s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.964701614s trace: https://jaeger.dev.storj.io/trace/fbd33bd94283fbe FAIL (2025-12-23 09:11:24 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.964729857s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 1.379501928s trace: https://jaeger.dev.storj.io/trace/292b9465c989993a FAIL (2025-12-23 09:11:22 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 1.379553846s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 2.576351001s trace: https://jaeger.dev.storj.io/trace/3727d7c571aed99f FAIL (2025-12-23 09:11:13 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.576393121s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 1.373493627s trace: https://jaeger.dev.storj.io/trace/292b9465c989993a FAIL (2025-12-23 09:11:02 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 1.373541547s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 14.081202889s trace: https://jaeger.dev.storj.io/trace/3b621a03316dcb85 FAIL (2025-12-23 09:11:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1NgPjVCyZZAfxVr3WvKrGRjnkpPtUvmQcCMiYntxNKVSWqXcHC, Piece ID: MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ) context deadline exceeded; piecestore: (Node ID: 14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj, Piece ID: SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ) context deadline exceeded; piecestore: (Node ID: 124taxpXdSnzEkq6UG1ZPi7cKLkaoDSc3G9FH1YHbGbDwMTM8PG, Piece ID: XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA) context deadline exceeded; piecestore: (Node ID: 121nTAu2qwnGorPfRFRJ7kE4hSsPq9p8zCZVFtt7cNNGu1HGdnN, Piece ID: KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA) context deadline exceeded ======LOGS====== 3358]: Uploading piece... Dec 23 08:47:14.746900773 (425h18m14.617983818s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.749939840 (425h18m14.621022885s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.753510229 (425h18m14.624593264s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.755262356 (425h18m14.626345391s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.756625842 (425h18m14.627708877s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.758025117 (425h18m14.629108172s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.760642110 (425h18m14.631725145s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.761429270 (425h18m14.632512315s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763241921 (425h18m14.634324966s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763973835 (425h18m14.63505688s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765312685 (425h18m14.63639573s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765648824 (425h18m14.636731869s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.767253601 (425h18m14.638336636s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.768200434 (425h18m14.639283469s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:16.245896991 (425h18m16.116980036s) [upload=373358]: Next returned nil segment Dec 23 08:47:16.245944181 (425h18m16.117027226s) [upload=373358]: Waiting for error group managing segments... Dec 23 08:47:16.644138110 (425h18m16.515221145s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.131664201 (425h18m17.002747236s) [noise=true , address=64.49.13.139:4475 , piece_id=TH3UWCMUNGGADRA26PZNKZLOWCLRLRWIT23ONF2B2DJ3NMMFPZBQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.444602902 (425h18m17.315685947s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.470421138 (425h18m17.341504173s) [noise=true , address=77.90.31.13:4630 , piece_id=SOJYUZH676PZYM4Z5YGEXW4ZXPAVLSEIFFZBE22LGAUOMRRKVG7A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.554527268 (425h18m17.425610313s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.059191355 (425h18m17.93027439s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.354549959 (425h18m18.225633004s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.436262252 (425h18m18.307345297s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.800612455 (425h18m18.67169549s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.973704980 (425h18m18.844788025s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.276001403 (425h18m19.147084438s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.339083580 (425h18m19.210166625s) [noise=true , address=77.90.31.19:4876 , piece_id=EO3PO3DW5CXWWO4T6ND7J4JTZYC3RI35MLMUEQXQVEABP63LQMQQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.349762695 (425h18m19.22084573s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636789835 (425h18m20.50787288s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636817878 (425h18m20.507900923s) [seg_pos={0 0}, upload=373358]: Segment reached optimal threshold of 14 pieces. Dec 23 08:47:20.637067674 (425h18m20.508150709s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf): protocol: expected piece hash; context canceled; EOF Dec 23 08:47:20.637077883 (425h18m20.508160938s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637240834 (425h18m20.508323879s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu): protocol: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection Dec 23 08:47:20.637249079 (425h18m20.508332125s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637346505 (425h18m20.50842956s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bFEmP78qGhyHhXr3QNVptdiWQ4No4nrb31qj6DPgmJLcHYKiM): protocol: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection Dec 23 08:47:20.637368196 (425h18m20.508451241s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637562967 (425h18m20.508646012s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1acQG8MjLvgzmCs8ELswcsKJrNvYmGQLX1FL4dHeCj2AUH7A5R): protocol: read tcp 188.166.225.170:56478->45.91.116.13:4170: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 23 08:47:20.637572605 (425h18m20.50865564s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637715547 (425h18m20.508798592s) [seg_pos={0 0}, upload=373358]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 23 08:47:20.637871194 (425h18m20.508954239s) [upload=373358]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 08:47:20.637878588 (425h18m20.508961633s) [upload=373358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 08:47:21.234004555 (425h18m21.10508763s) [upload=373358]: Done uploading segments... Dec 23 08:47:21.234072815 (425h18m21.10515586s) [upload=373358]: Upload finished. err: <nil> Duration: 14.08122946s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.275529275s trace: https://jaeger.dev.storj.io/trace/265f3259a7705b6d FAIL (2025-12-23 09:10:58 UTC) orbiter warning: operation: sync2.Copy uplink: metaclient: context deadline exceeded Duration: 5.27555797s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.058384667s trace: https://jaeger.dev.storj.io/trace/265f3259a7705b6d FAIL (2025-12-23 09:10:40 UTC) orbiter warning: operation: sync2.Copy uplink: metaclient: context deadline exceeded Duration: 5.058411699s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 8.927116574s trace: https://jaeger.dev.storj.io/trace/6156a7d944a046d FAIL (2025-12-23 09:10:29 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 8.927139518s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.297818061s trace: https://jaeger.dev.storj.io/trace/3c39be5f434273aa FAIL (2025-12-23 09:10:29 UTC) orbiter warning: context deadline exceeded Duration: 6.297838229s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.87914503s trace: https://jaeger.dev.storj.io/trace/fbd33bd94283fbe FAIL (2025-12-23 09:10:22 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.879175208s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 26.078926333s trace: https://jaeger.dev.storj.io/trace/3b621a03316dcb85 FAIL (2025-12-23 09:10:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1NgPjVCyZZAfxVr3WvKrGRjnkpPtUvmQcCMiYntxNKVSWqXcHC, Piece ID: MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ) context deadline exceeded; piecestore: (Node ID: 14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj, Piece ID: SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ) context deadline exceeded; piecestore: (Node ID: 124taxpXdSnzEkq6UG1ZPi7cKLkaoDSc3G9FH1YHbGbDwMTM8PG, Piece ID: XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA) context deadline exceeded; piecestore: (Node ID: 121nTAu2qwnGorPfRFRJ7kE4hSsPq9p8zCZVFtt7cNNGu1HGdnN, Piece ID: KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA) context deadline exceeded ======LOGS====== 3358]: Uploading piece... Dec 23 08:47:14.746900773 (425h18m14.617983818s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.749939840 (425h18m14.621022885s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.753510229 (425h18m14.624593264s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.755262356 (425h18m14.626345391s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.756625842 (425h18m14.627708877s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.758025117 (425h18m14.629108172s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.760642110 (425h18m14.631725145s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.761429270 (425h18m14.632512315s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763241921 (425h18m14.634324966s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763973835 (425h18m14.63505688s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765312685 (425h18m14.63639573s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765648824 (425h18m14.636731869s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.767253601 (425h18m14.638336636s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.768200434 (425h18m14.639283469s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:16.245896991 (425h18m16.116980036s) [upload=373358]: Next returned nil segment Dec 23 08:47:16.245944181 (425h18m16.117027226s) [upload=373358]: Waiting for error group managing segments... Dec 23 08:47:16.644138110 (425h18m16.515221145s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.131664201 (425h18m17.002747236s) [noise=true , address=64.49.13.139:4475 , piece_id=TH3UWCMUNGGADRA26PZNKZLOWCLRLRWIT23ONF2B2DJ3NMMFPZBQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.444602902 (425h18m17.315685947s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.470421138 (425h18m17.341504173s) [noise=true , address=77.90.31.13:4630 , piece_id=SOJYUZH676PZYM4Z5YGEXW4ZXPAVLSEIFFZBE22LGAUOMRRKVG7A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.554527268 (425h18m17.425610313s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.059191355 (425h18m17.93027439s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.354549959 (425h18m18.225633004s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.436262252 (425h18m18.307345297s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.800612455 (425h18m18.67169549s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.973704980 (425h18m18.844788025s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.276001403 (425h18m19.147084438s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.339083580 (425h18m19.210166625s) [noise=true , address=77.90.31.19:4876 , piece_id=EO3PO3DW5CXWWO4T6ND7J4JTZYC3RI35MLMUEQXQVEABP63LQMQQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.349762695 (425h18m19.22084573s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636789835 (425h18m20.50787288s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636817878 (425h18m20.507900923s) [seg_pos={0 0}, upload=373358]: Segment reached optimal threshold of 14 pieces. Dec 23 08:47:20.637067674 (425h18m20.508150709s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf): protocol: expected piece hash; context canceled; EOF Dec 23 08:47:20.637077883 (425h18m20.508160938s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637240834 (425h18m20.508323879s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu): protocol: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection Dec 23 08:47:20.637249079 (425h18m20.508332125s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637346505 (425h18m20.50842956s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bFEmP78qGhyHhXr3QNVptdiWQ4No4nrb31qj6DPgmJLcHYKiM): protocol: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection Dec 23 08:47:20.637368196 (425h18m20.508451241s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637562967 (425h18m20.508646012s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1acQG8MjLvgzmCs8ELswcsKJrNvYmGQLX1FL4dHeCj2AUH7A5R): protocol: read tcp 188.166.225.170:56478->45.91.116.13:4170: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 23 08:47:20.637572605 (425h18m20.50865564s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637715547 (425h18m20.508798592s) [seg_pos={0 0}, upload=373358]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 23 08:47:20.637871194 (425h18m20.508954239s) [upload=373358]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 08:47:20.637878588 (425h18m20.508961633s) [upload=373358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 08:47:21.234004555 (425h18m21.10508763s) [upload=373358]: Done uploading segments... Dec 23 08:47:21.234072815 (425h18m21.10515586s) [upload=373358]: Upload finished. err: <nil> Duration: 26.078944217s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 1.432666943s trace: https://jaeger.dev.storj.io/trace/292b9465c989993a FAIL (2025-12-23 09:10:07 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 1.432697272s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 1.943366366s trace: https://jaeger.dev.storj.io/trace/3727d7c571aed99f FAIL (2025-12-23 09:10:06 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.943439434s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.488062838s trace: https://jaeger.dev.storj.io/trace/fbd33bd94283fbe FAIL (2025-12-23 09:09:59 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 6.488081793s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.749388471s trace: https://jaeger.dev.storj.io/trace/3c39be5f434273aa FAIL (2025-12-23 09:09:57 UTC) orbiter warning: context deadline exceeded Duration: 5.749418698s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 20.080590871s trace: https://jaeger.dev.storj.io/trace/3b621a03316dcb85 FAIL (2025-12-23 09:09:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1NgPjVCyZZAfxVr3WvKrGRjnkpPtUvmQcCMiYntxNKVSWqXcHC, Piece ID: MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ) context deadline exceeded; piecestore: (Node ID: 14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj, Piece ID: SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ) context deadline exceeded; piecestore: (Node ID: 124taxpXdSnzEkq6UG1ZPi7cKLkaoDSc3G9FH1YHbGbDwMTM8PG, Piece ID: XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA) context deadline exceeded; piecestore: (Node ID: 121nTAu2qwnGorPfRFRJ7kE4hSsPq9p8zCZVFtt7cNNGu1HGdnN, Piece ID: KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA) context deadline exceeded ======LOGS====== 3358]: Uploading piece... Dec 23 08:47:14.746900773 (425h18m14.617983818s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.749939840 (425h18m14.621022885s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.753510229 (425h18m14.624593264s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.755262356 (425h18m14.626345391s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.756625842 (425h18m14.627708877s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.758025117 (425h18m14.629108172s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.760642110 (425h18m14.631725145s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.761429270 (425h18m14.632512315s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763241921 (425h18m14.634324966s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.763973835 (425h18m14.63505688s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765312685 (425h18m14.63639573s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.765648824 (425h18m14.636731869s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.767253601 (425h18m14.638336636s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:14.768200434 (425h18m14.639283469s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Uploading piece... Dec 23 08:47:16.245896991 (425h18m16.116980036s) [upload=373358]: Next returned nil segment Dec 23 08:47:16.245944181 (425h18m16.117027226s) [upload=373358]: Waiting for error group managing segments... Dec 23 08:47:16.644138110 (425h18m16.515221145s) [noise=true , address=145.79.227.3:4171 , piece_id=2ISHFXQR7VICM3SDXJ4HG5TF7AYMVHAVKYUUQ3XQI7NQKI6PZLLQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.131664201 (425h18m17.002747236s) [noise=true , address=64.49.13.139:4475 , piece_id=TH3UWCMUNGGADRA26PZNKZLOWCLRLRWIT23ONF2B2DJ3NMMFPZBQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.444602902 (425h18m17.315685947s) [noise=true , address=145.79.227.49:4352 , piece_id=GQK36CQOHIY4FYIMUAVINQBLS2WYMFSEPNG34RKKG2ZBD544DLTA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.470421138 (425h18m17.341504173s) [noise=true , address=77.90.31.13:4630 , piece_id=SOJYUZH676PZYM4Z5YGEXW4ZXPAVLSEIFFZBE22LGAUOMRRKVG7A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:17.554527268 (425h18m17.425610313s) [noise=true , address=145.79.227.30:4772 , piece_id=KZDCAC4ZBLROH4YV3ANRYCSHLZ7OJHWWUIY46ISTW2HHLCLSR53Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.059191355 (425h18m17.93027439s) [noise=true , address=77.90.31.21:4893 , piece_id=MNIKI3ZCNWZLWZ3TXYRADGNVXMIAMRSZZLJCOTDC6JEYTPVKZXFQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.354549959 (425h18m18.225633004s) [noise=true , address=145.79.227.44:4208 , piece_id=YHWJ6CSFQ2RPBR4B5N4GOAMWRIMFEJMCSUR6QOBFMP6D3R3AMZGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.436262252 (425h18m18.307345297s) [noise=true , address=77.90.31.14:4689 , piece_id=4APHUPZVMUIEVO4OLQJVZEMUGBAN52JH352LODHJOWTTBUPWGPEA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.800612455 (425h18m18.67169549s) [noise=true , address=5.231.73.5:4009 , piece_id=KJTR5KHSIYZKDLZJPDBTOU6X7HG7PEMW64NPXDMMKW7OLMKKSWQA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:18.973704980 (425h18m18.844788025s) [noise=true , address=77.90.31.11:4586 , piece_id=R6JMMBJQBYTSUNIISYRZIJEE5HF6CFIZ6T7TDVTS2JAEMV6JGX4A, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.276001403 (425h18m19.147084438s) [noise=true , address=45.91.116.10:4108 , piece_id=XHUTQNL3KUOF6TMLX5OMCWSVP3TKCO2R27P66C4RMIHZUTJIB6BA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.339083580 (425h18m19.210166625s) [noise=true , address=77.90.31.19:4876 , piece_id=EO3PO3DW5CXWWO4T6ND7J4JTZYC3RI35MLMUEQXQVEABP63LQMQQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:19.349762695 (425h18m19.22084573s) [noise=true , address=77.90.31.15:4704 , piece_id=SOECN5LE7ZVW6Y22V26SRP2PS52RSTPMW22QQNIV3TR72Z7VI7VQ, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636789835 (425h18m20.50787288s) [noise=true , address=45.91.116.4:4963 , piece_id=5VBEH5PT3TB2MFIERPIUJRWOXG7BZ5W7IY7DDB52KMQPC6WXZB6Q, seg_pos={0 0}, upload=373358]: Done uploading piece. err: <nil> Dec 23 08:47:20.636817878 (425h18m20.507900923s) [seg_pos={0 0}, upload=373358]: Segment reached optimal threshold of 14 pieces. Dec 23 08:47:20.637067674 (425h18m20.508150709s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf): protocol: expected piece hash; context canceled; EOF Dec 23 08:47:20.637077883 (425h18m20.508160938s) [noise=true , address=77.90.31.12:4602 , piece_id=JQUMVMVQ2XDQLP7WMXGQ4Q3WXN54XPR47KNRZ4XTU7R67KQ65XGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637240834 (425h18m20.508323879s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nxT1Ldr46HPrDRHQ2nrwEBZT5bEVoPAAfBz86ZYwUH13sqwHu): protocol: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47330->64.49.13.169:4711: use of closed network connection Dec 23 08:47:20.637249079 (425h18m20.508332125s) [noise=true , address=64.49.13.169:4711 , piece_id=HWAE2EG7HNSLR2QHDEV6MGDQB4WAUNNRTFVEHLOUFV23HBRUWGGA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637346505 (425h18m20.50842956s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bFEmP78qGhyHhXr3QNVptdiWQ4No4nrb31qj6DPgmJLcHYKiM): protocol: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43634->77.90.31.17:4791: use of closed network connection Dec 23 08:47:20.637368196 (425h18m20.508451241s) [noise=true , address=77.90.31.17:4791 , piece_id=Q5J37SE6XBUANPYD5GRXBSSN43FRYWQDVH3Q6JMAGYAHGG2STQBA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637562967 (425h18m20.508646012s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1acQG8MjLvgzmCs8ELswcsKJrNvYmGQLX1FL4dHeCj2AUH7A5R): protocol: read tcp 188.166.225.170:56478->45.91.116.13:4170: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 23 08:47:20.637572605 (425h18m20.50865564s) [noise=true , address=45.91.116.13:4170 , piece_id=I6FP3YTSKFBTZHL6G45VY57FKRG3KGVUKDO55UED5TSNC4VE62IA, seg_pos={0 0}, upload=373358]: Long tail cancellation due to optimal threshold Dec 23 08:47:20.637715547 (425h18m20.508798592s) [seg_pos={0 0}, upload=373358]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 23 08:47:20.637871194 (425h18m20.508954239s) [upload=373358]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 23 08:47:20.637878588 (425h18m20.508961633s) [upload=373358]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 23 08:47:21.234004555 (425h18m21.10508763s) [upload=373358]: Done uploading segments... Dec 23 08:47:21.234072815 (425h18m21.10515586s) [upload=373358]: Upload finished. err: <nil> Duration: 20.080618465s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.987325894s trace: https://jaeger.dev.storj.io/trace/6156a7d944a046d FAIL (2025-12-23 09:09:40 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 6.987354127s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.154149108s trace: https://jaeger.dev.storj.io/trace/265f3259a7705b6d FAIL (2025-12-23 09:09:35 UTC) orbiter warning: operation: sync2.Copy uplink: metaclient: context deadline exceeded Duration: 5.154179706s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.979807247s trace: https://jaeger.dev.storj.io/trace/fbd33bd94283fbe FAIL (2025-12-23 09:09:33 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.979837756s core uplink us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)